视频防抖预研（二）—— 数据采集

代码仓库：https://github.com/xiaoqinxing/EIS_data_request_app

规划

第一个方案是从camera底层获取录像和陀螺仪数据。

问题1：我们一直使用的是预览流数据，根本不知道从什么时候开始录像，录像和拍照只是媒控取得预览流数据进行编码得到的。
问题2：很难将每帧图像对应的陀螺仪数据采集下来。

第二个方案是用Java做一个APP去实现陀螺仪和图像数据的采集
毫无疑问，我们选择第二种方案。

实现

（一）权限申请

1. 安卓6.0之前

<uses-permission android:name="android.permission.CAMERA" />
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />

放在xml中申请权限，位置在application之前

2. 安卓6.0之后

权限需要动态申请：在APP初始化的时候，进行权限的申请。本文引用了一个权限的工具类，具体的内容可以查看源码。

（二）实现预览及录像功能

1. 修改界面布局xml

添加一个textureView作为预览画面的显示
添加一个button控制录像的开关

2. 初始化（去除申请权限的代码）

public class Recorder extends AppCompatActivity {
    private TextureView mPreview;  // For displaying the live camera preview
    private Camera mCamera;        // Object to contact the camera hardware
    private MediaRecorder mMediaRecorder;    // Store the camera's image stream as a video

    private boolean isRecording = false; // Is video being recoded
    private Button btnRecord;            // Button that triggers recording

    private static String TAG = "GyroRecorder";

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        //显示一个页面
        setContentView(R.layout.activity_main);

        //id is elements's id to find which elements you choose
        mPreview = (TextureView)findViewById(R.id.textureView1);
        btnRecord = (Button)findViewById(R.id.button1);

        btnRecord.setOnClickListener(new View.OnClickListener() {
            @Override
            public void onClick(View view) {
                onCaptureClick(view);
            }
        });
    }
}

3. 处理按键按下的逻辑

当没有录像时，按下按键就创建一个新的线程，启动录像。在启动录像的之后，把按键上的名称改成“Stop”
当正在录像的时候，就停止录像并释放资源

public void onCaptureClick(View view) {
    if (isRecording) {
        // Already recording? Release camera lock for others
        mMediaRecorder.stop();
        releaseMediaRecorder();
        mCamera.lock();

        isRecording = false;
        releaseCamera();
        mGyroFile.close();
        mGyroFile = null;
        btnRecord.setText("Start");
        mStartTime = -1;
    } else {
        // Not recording – launch new "thread" to initiate!
        new MediaPrepareTask().execute(null, null, null);
    }
}

class MediaPrepareTask extends AsyncTask<Void, Void, Boolean> {
    //automatically creates a new thread and runs doInBackground in that thread
    @Override
    protected Boolean doInBackground(Void... voids) {
        
        //identifying supported image sizes from the camera, finding the suitable height,
        // setting the bitrate of the video and specifying the destination video file.
        if(prepareVideoRecorder()) {
            mMediaRecorder.start();
            isRecording = true;
        } else {
            releaseMediaRecorder();
            return false;
        }
        return true;
    }

    @Override
    protected void onPostExecute(Boolean result) {
        if(!result) {
            Recorder.this.finish();
        }

        btnRecord.setText("Stop");
    }
}

// 释放recorder
private void releaseMediaRecorder() {
    if(mMediaRecorder != null) {
        mMediaRecorder.reset();
        mMediaRecorder.release();
        mMediaRecorder = null;
        mCamera.lock();
    }
}
//释放camera
private void releaseCamera() {
    if(mCamera != null) {
        mCamera.release();
        mCamera = null;
    }
}

4. 打开预览及开始录像

打开相机
查找并设置camera预览的尺寸
开启camera的预览，并通过textureView显示出来
查找并设置camera录像的尺寸、格式、输出路径等参数

@TargetApi(Build.VERSION_CODES.HONEYCOMB)
private boolean prepareVideoRecorder() {
    // because mCamera is privated object, so other class can't used
    mCamera = Camera.open();
    if (mCamera == null) {
        Toast.makeText(this, "没有可用相机", Toast.LENGTH_SHORT).show();
        return false;
    }

    Camera.Parameters parameters = mCamera.getParameters();
    List mSupportedPreviewSizes = parameters.getSupportedPreviewSizes();

    // find the optimal image size for the camera
    Camera.Size optimalSize = getOptimalPreviewSize(mSupportedPreviewSizes,mPreview.getWidth(),mPreview.getHeight());
    parameters.setPreviewSize(optimalSize.width,optimalSize.height);

    //With the optimal size in hand, we can now set up the camera recorder settings
    CamcorderProfile profile = CamcorderProfile.get(CamcorderProfile.QUALITY_HIGH);

    //contact the camera hardware and set up these parameters
    mCamera.setParameters(parameters);
    mCamera.startPreview();

    try {
        mCamera.setPreviewTexture(mPreview.getSurfaceTexture());
    } catch(IOException e) {
        Log.e(TAG,"Surface texture is unavailable or unsuitable" + e.getMessage());
        return false;
    }
    //camera unlock need before new a MediaRecorder object
    mCamera.unlock();
    //set up the media recorder
    mMediaRecorder = new MediaRecorder();

    mMediaRecorder.setCamera(mCamera);

    mMediaRecorder.setVideoSource(MediaRecorder.VideoSource.CAMERA);

    mMediaRecorder.setOutputFormat(profile.fileFormat);

    //maybe video sizes are different from preview stream, so get max supported video sizes
    List supportedVideoSizes = parameters.getSupportedVideoSizes();
    for(int i=0;i
        Log.d("init", "supportedVideoSize:"+supportedVideoSizes.get(i).height+
                "x"+supportedVideoSizes.get(i).width);
    }
    profile.videoFrameWidth = supportedVideoSizes.get(0).width;
    profile.videoFrameHeight = supportedVideoSizes.get(0).height;
    mMediaRecorder.setVideoSize(profile.videoFrameWidth,profile.videoFrameHeight);

    mMediaRecorder.setVideoEncodingBitRate(profile.videoBitRate);

    mMediaRecorder.setVideoEncoder(profile.videoCodec);
    mMediaRecorder.setOutputFile(getOutputMediaFile().toString());

    //initialize the PrintStream
    try {
        mGyroFile = new PrintStream(getOutputGyroFile());
        mGyroFile.append("gyro\n");
    } catch(IOException e) {
        Log.d(TAG, "Unable to create acquisition file");
        return false;
    }

    try {
        mMediaRecorder.prepare();
    } catch (IllegalStateException e) {
        Log.d(TAG, "IllegalStateException preparing MediaRecorder: " + e.getMessage());
        releaseMediaRecorder();
        return false;
    } catch (IOException e) {
        Log.d(TAG, "IOException preparing MediaRecorder: " + e.getMessage());
        releaseMediaRecorder();
        return false;
    }
    return true;
}

（三）添加传感器数据

当录像的时候开始记录陀螺仪的数据，主要记录陀螺仪的值以及对应的时间戳

1. 初始化

//The SensorManager object manages all sensors on the hardware
private SensorManager mSensorManager;
private Sensor mGyro;
private PrintStream mGyroFile;
private long mStartTime = -1;
private List sensorList;
mSensorManager = (SensorManager)getSystemService(Context.SENSOR_SERVICE);

@Override
protected void onCreate(Bundle savedInstanceState) {
    ...
    //print all sensor's name
    List sensorNameList = new ArrayList();
    sensorList = mSensorManager.getSensorList(Sensor.TYPE_ALL);
    for(Sensor sensor : sensorList) {
        sensorNameList.add(sensor.getName()+"\r\n");
    }
    Log.d("sensor",sensorNameList.toString());

    //register sensor listener for gyro
    mGyro = mSensorManager.getDefaultSensor(Sensor.TYPE_GYROSCOPE);
    //fetching the gyroscope sensor and registering that this class should receive events (registerListener)
    mSensorManager.registerListener(gyro_listener, mGyro, SensorManager.SENSOR_DELAY_FASTEST);
}

2. 数据打印流初始化

//initialize the PrintStream
try {
    //获取陀螺仪文件输出路径
    mGyroFile = new PrintStream(getOutputGyroFile());
    mGyroFile.append("gyro\n");
} catch(IOException e) {
    Log.d(TAG, "Unable to create acquisition file");
    return false;
}
``` 

#### 3. 监听传感器数据
``` java
private SensorEventListener gyro_listener = new SensorEventListener(){
    @Override
    public void onAccuracyChanged(Sensor sensor, int accuracy) {
        // Empty on purpose
        // Required because we implement SensorEventListener
    }

    @Override
    public void onSensorChanged(SensorEvent sensorEvent) {
        if(isRecording) {
            if(mStartTime == -1) {
                mStartTime = sensorEvent.timestamp;
            }
            mGyroFile.append(sensorEvent.values[0] + "," +
                    sensorEvent.values[1] + "," +
                    sensorEvent.values[2] + "," +
                    (sensorEvent.timestamp-mStartTime) + "\n");
        }
    }
};