Use FFmpeg instead of EmguCV

[Speykious]

Currently, EmguCV is being used only to handle webcam input. We've had various problems with runtimes not being in the right place and cameras not being detected.

Thus I propose that we use FFmpeg for that task. I think that it will be much easier to deal with as we can just use it as a system-installed binary. Not to mention that the library is LGPL which is just perfect for our use-case.

[LeNitrous]

Please provide details on how you plan to implement it as well.

[Speykious]

From what I understand, all we need is to get the raw bytes of the image from the webcam.

Then you can refer to https://trac.ffmpeg.org/wiki/Capture/Webcam. It would be as simple as reading the stdout from a command along the lines of

# Windows
ffmpeg -f vfwcap -i $device_index -pix_fmt rgba -f rawvideo -
# Linux
ffmpeg -f v4l2 -i /dev/video$device_index -pix_fmt rgba -f rawvideo -

to take raw RGBA pixels from the desired webcam. We can even introduce an -input_format argument in case a device supports several input formats, for example on my machine:

❯ v4l2-ctl -d 0 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
        Type: Video Capture

        [0]: 'MJPG' (Motion-JPEG, compressed)
                Size: Discrete 1280x720
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 640x480
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 640x360
                        Interval: Discrete 0.033s (30.000 fps)
        [1]: 'YUYV' (YUYV 4:2:2)
                Size: Discrete 1280x720
                        Interval: Discrete 0.100s (10.000 fps)
                Size: Discrete 640x480
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 640x360
                        Interval: Discrete 0.033s (30.000 fps)

You can test that the ffmpeg command above that outputs raw pixels works on Windows with

ffmpeg -f vfwcap -i 0 -pix_fmt rgba -f rawvideo - | ffmpeg -f rawvideo -pix_fmt rgba -s $sizeofwebcamhere -i - camerout.mkv

then cancel the command after a bit and open camerout.mkv in a video player. Change the device number if you have several of them.

[sr229]

per the Discord discussion, we're re-assigning this to Encore 1 instead since this is more of something we wanna fix as Encore 1 is a stable client.

[Speykious]

As Vignette Debut has been released, I and @adryzz have begun to engage in that task. We will create a small and agnostic library using FFmpeg.Autogen, which is a one-to-one mapping of the FFmpeg C API in C#, as it is preferable to use the low level API rather than the command line tool.

The lack of documentation doesn't make this task easy, but thankfully, most of the important stuff has been figured out.

[Speykious]

Yay! SeeShark exists now.