`VideoReader.record_stream()` doesn't handle the cases when processing/preprocessing change the frame size

[MatusGasparik]

I'd like to like to stream a video from a file, select a specific ROI, apply some functions (predictions from a model) and save the result in another file.

I tried to use the record_stream() but had no success saving an output file (the file was actually saved but it was corrupt).

The problem seems specific to the case when the (pre)-process functions affect the dimensions of the video (e.g. crop):

from functools import partial
from pcv.vidIO import VideoReader

rois = [
    (( 0.0, 0.0), 0.5, 0.5),
    (( 0.0, 0.5), 0.5, 0.5),
    ((0.5, 0.25), 0.5, 0.5),
]

def crop(frame, roi):
    H, W = frame.shape[:2]
    (y, x), h, w = roi
    h = int(H * h)
    w = int(W * w)
    y = int(H * y)
    x = int(W * x)

    return frame[y:y+h, x:x+w]

with VideoReader("myvid.mp4", process=partial(crop, roi=rois[1])) as vid:
    vid.record_stream("result.mp4")     # outputs a corrupted file

The results on the screen are o.k.

This was probably not the intended use. But I need to process a "patchwork" of videos in a single video...

I haven't tried yet, but can remedy this by specifying the writer argument of the record_stream() function with appropriate WIDTH and HEIGHT parameters?

[MatusGasparik]

I was too quick opening this issue... Of course this can't work. But maybe this use case can serve as a basis for some future enhancement where it is possible to specify a ROI in the VideoReader initializer? 😉

[ES-Alexander]

The record_stream method by default uses the VideoWriter.from_camera method to create the writer. If no frameSize keyword argument is passed in, then the input stream is queried for its (width, height) as the default size to try to output.

That default behaviour could potentially be made more intelligent by getting the size of the first frame, but that complicates the logic, particularly in cases where it's important that every input frame gets processed and returned as an output frame. To override the default you can just specify that argument though :-)

Given this issue has been raised, that's a pretty good sign that I should document that the record_stream kwargs get passed to the writer's from_camera method, so that at least it would tell you where to look if you used help(VideoReader) (or help(Camera), etc) instead of needing to look at the code. I've documented several of the other kwargs, so not sure why that one was missed.

---

Note that the parameter uses lowerCamelCase for compatibility/easier changeover with code that's using the default OpenCV cv2.VideoWriter.