[from SO|kesh] inquiry

[tikuma-lsuhsc]

Hey @antortjim, this is kesh from SO,

This is a solicitation message, but if you find it useful, I'd be thrilled.

I've been developing a Python FFmpeg package called ffmpegio mostly for my research use (analyses of human voice disorders), but I'm hoping to find others who would use it. It seems like your FFmpeg use case with camera recording matches up well. Below is a quick take on how it could be used with pypylon

from pypylon import pylon
import ffmpegio

camera = pylon.InstantCamera(pylon.TlFactory.GetInstance().CreateFirstDevice())
camera.Open()

# camera setup goes here

with ffmpegio.open(
    "output.mp4", # output file
    "wv",           # mode: video write
    rate_in=45,  # input video framerate
    vsync_in=0, # not sure you need this, but specify input FFmpeg options with '_in' appendix (drop '-' prefix )
    extra_hw_frames_in=2,
    overwrite=True, # sets '-y' flag
    vcodec="h264_nvenc", # output option, just drop '-' prefix.
) as mp4:
    while camera.IsGrabbing():
        grabResult = camera.RetrieveResult(5000, pylon.TimeoutHandling_ThrowException)
        if grabResult.GrabSucceeded():
            img = (
                grabResult.GetArrayZeroCopy()
            )  # or GetArray() ?
            # Make sure img shape is [nframes x h x w x ncomp] If grayscale, could be [nframes x h x w]

            mp4.write(img)
            # `-s` and `-pix_fmt` FFmpeg options will be set according to the first frame written

        grabResult.Release()
camera.Close()

Thanks for reading! -Kesh

[antortjim]

Hi @tikuma-lsuhsc ! Thanks for your comment I have

1. added the camera.StartGrabbing line, otherwise the for loop is never entered (i.e. Open() alone is not enough)
2. replaced GetArrayZeroCopy with just GetArray because the output of ZeroCopy is not an array, nor is it subscribable in my setup. Which is weird, given this example https://github.com/basler/pypylon/blob/master/samples/zerocopy.py

With this code I am able to achieve the same performance as with my code in this repo. I think it's cleaner so I will try to implement yours. Even though I need to double check it is equally quick to write. Thank you anyway!

PS Attached the log of reading and writing times from the Basler camera with resolution 3840x2160 in gray color benchmark.log

from pypylon import pylon
import ffmpegio

camera = pylon.InstantCamera(pylon.TlFactory.GetInstance().CreateFirstDevice())
camera.Open()
camera.PixelFormat = "Mono8"
#camera.StartGrabbing(pylon.GrabStrategy_LatestImageOnly)
camera.StartGrabbing(pylon.GrabStrategy_OneByOne)

# camera setup goes here
with ffmpegio.open(
    "output.mp4", # output file
    "wv",           # mode: video write
    rate_in=45,  # input video framerate
    vsync_in=0, # not sure you need this, but specify input FFmpeg options with '_in' appendix (drop '-' prefix )
    extra_hw_frames_in=2,
    overwrite=True, # sets '-y' flag
    vcodec="h264_nvenc", # output option, just drop '-' prefix.
) as mp4:

    while camera.IsGrabbing():
        grabResult = camera.RetrieveResult(5000, pylon.TimeoutHandling_ThrowException)
        if grabResult.GrabSucceeded():
            img = (
                grabResult.GetArray()
            )  # or GetArray() ?
            # Make sure img shape is [nframes x h x w x ncomp] If grayscale, could be [nframes x h x w]

            mp4.write(img)
            # `-s` and `-pix_fmt` FFmpeg options will be set according to the first frame written
        else:
            logging.warning("Grab failure")

        grabResult.Release()
camera.Close()

[tikuma-lsuhsc]

Oh cool. Thanks you for giving it a try. Just FYI, open with mode='vw' uses the SimpleVideoWriter class in streams.SimpleVideoWriter, with SimpleWriterBase.write() which essentially does nothing but write to pipe's stdin once FFmpeg starts rolling. (There is an additional cost of implementing the plugin system but hopefully that's minimum.)

[tikuma-lsuhsc]

Did adding camera.PixelFormat = "Mono8" help lowering CPU utilization? Or more specifically what's the dimension of img?