search

EasternEdgeRobotics / Software_2017

The control software for 2017
MIT License
3 stars 0 forks source link

Faster camera startup and the return of HD stills #313

Closed cal-pratt closed 7 years ago

cal-pratt commented 7 years ago

In this PR I have reduced the startup time of the camera's from about 20 seconds to roughly 6 seconds. I also added a new way to take the HD screenshots which does not cause a FOV reduction.

Faster Startup

It turns out a lot of the startup time is caused by the topside decoder being initially starved of frames. To remedy this, once the connection is established I feed the decoder a dummy video, 20 seconds long of 20 FPS H264 frames. 

final FrameGrabber grabber = new FFmpegFrameGrabber(
    new SequenceInputStream(
        new FileInputStream("dummyvideo.h264"),
        clientSocket.getInputStream()));

Very surprised how well this worked and it also adds some style to GUI. The dummy video I created was just the logo, but eventually someone could make a cooler one if we have time.

image

Still frames

The last attempt at high resolution still frames was to run a base stream at a higher resolution and resizing the stream which gets sent to the topside. Still images are then captured from the base stream. In this solution I could only increase the image resolution marginally (720 to 1080p) before the pi couldn't handle the resize task anymore. Unfortunately 1080p appears to be a greatly reduced FOV. Further, 720p appears to also be reduced. We can investigate the 4:3 1640x1232 mode.

http://elinux.org/Rpi_Camera_Module#Technical_Parameters_.28v.2_board.29

Video Modes: 1 - 1080P30 cropped (680 pixels off left/right, 692 pixels off top/bottom), up to 30fps 2 - 3240x2464 Full 4:3, up to 15fps 3 - 3240x2464 Full 4:3, up to 15fps (identical to 2) 4 - 1640x1232 binned 4:3, up 40fps 5 - 1640x922 2x2 binned 16:9 (310 px crop T/B before binning), up to 40fps 6 - 720P bin+crop (360 px L/R, 512 px T/B before binning), 40..90fps (OC: 120fps) 7 - VGA bin+crop (1000 px L/R, 752 px T/B before binning), 40..90fps (OC: 120fps)

Anyways, back on topic... In this solution I have created a process which allows us to take a still at any resolution we'd like, independant of the stream resolution. I stop the video recording process altogether using stop_recording, change the resolution to a higher quality, take an image, send that image to the topside, revert the resolution change, and then start a new video stream using start_recording. The calls to stop_recording do not actually close the file object they are writing to, which means our connection to the video decoder will stay open.

This process knocks out the camera for about 10 seconds (resolution changes and stream reconnect), but I believe that is a reasonable sacrifice-- meanwhile the pilot will be able to use the other camera for going to the next location.

cal-pratt commented 7 years ago

Buffer video needs to be calibrated for speed of the topside computer. Should consider making the file shorter and cascading SequenceInputStreams:

InputStream dummy =  new FileInputStream("dummyvideo.h264");
for (int i = 1; i < config.bufferCount(); i++) {
    dummy = new SequenceInputStream(new FileInputStream("dummyvideo.h264"), dummy));
}