Investigate lossless compressions

[yarikoptic]

https://antumdeluge.wordpress.com/lossless-ffmpeg-video-encoders/

We might want to grab using one, performance efficient, then recompress using the most appropriate for a given video

[yarikoptic]

FWIW, tried on a small sample file with blindly going through a few compressors:

f=2020.03.05.15.18.34_2020.03.05.15.18.43.mkv; for vcodec in ffv1 flashsv gif png qtrle zmbv ffvhuff huffyuv ljpeg r10k; do time ffmpeg -i "$f" -vcodec $vcodec "${f//.mkv/-$vcodec.mkv}"; done

and also manual

ffmpeg -i 2020.03.05.15.18.34_2020.03.05.15.18.43.mkv -c:v libx264 -preset veryslow -crf 0 2020.03.05.15.18.34_2020.03.05.15.18.43-libx264-veryslow-crf0.mkv

and the original one beats them all

$> ls -lS 2020.03.05.15.18.34_2020.03.05.15.18.43*                                                                                                   
-rw------- 1 yoh yoh 2654234059 Mar  5 16:10 2020.03.05.15.18.34_2020.03.05.15.18.43-r10k.mkv
-rw------- 1 yoh yoh  350986442 Mar  5 16:10 2020.03.05.15.18.34_2020.03.05.15.18.43-ffvhuff.mkv
-rw------- 1 yoh yoh  350986442 Mar  5 16:10 2020.03.05.15.18.34_2020.03.05.15.18.43-huffyuv.mkv
-rw------- 1 yoh yoh  114410113 Mar  5 16:08 2020.03.05.15.18.34_2020.03.05.15.18.43-png.mkv
-rw------- 1 yoh yoh   54690284 Mar  5 16:08 2020.03.05.15.18.34_2020.03.05.15.18.43-flashsv.mkv
-rw------- 1 yoh yoh   48942534 Mar  5 16:08 2020.03.05.15.18.34_2020.03.05.15.18.43-ffv1.mkv
-rw------- 1 yoh yoh   41958745 Mar  5 16:08 2020.03.05.15.18.34_2020.03.05.15.18.43-qtrle.mkv
-rw------- 1 yoh yoh   13282631 Mar  5 16:10 2020.03.05.15.18.34_2020.03.05.15.18.43-zmbv.mkv
-rw------- 1 yoh yoh    3933457 Mar  5 16:08 2020.03.05.15.18.34_2020.03.05.15.18.43-gif.mkv
-rw------- 1 yoh yoh    2227212 Mar  5 16:10 2020.03.05.15.18.34_2020.03.05.15.18.43-libx264-veryslow-crf0.mkv
-rw------- 1 yoh yoh     293398 Mar  5 15:54 2020.03.05.15.18.34_2020.03.05.15.18.43.mkv

so the (original) AVC is the unquestionable "winner", but I don't think it was lossless when I look at the video and since explicit attempt to compress lossless using AVC resulted in x10 larger file. So -- we need to check either compression is done in hardware and ffmpeg just dumps it (then there could be some API to change codec/settings) or it compresses on the fly (and thus we would need to adjust ffmpeg invocation)

[andycon]

My understanding is that mpeg-4 (AVC) aks (H.264) is a state of the art lossy codec. What you appear to have done above is t o convert from a lossy recording to some other compression. But is we want to do lossless encoding, we need to do the initial encoding with a lossless codec. FFV1 is lossless. But my hunch is that mpeg-4 may be sufficient even if its is lossy. A good quality lossy video that is small may be better that a lossless video that is too large.. https://trac.ffmpeg.org/wiki/Encode/FFV1

On Thu, Mar 5, 2020 at 4:24 PM Yaroslav Halchenko notifications@github.com wrote:

FWIW, tried on a small sample file with blindly going through a few compressors:

f=2020.03.05.15.18.34_2020.03.05.15.18.43.mkv; for vcodec in ffv1 flashsv gif png qtrle zmbv ffvhuff huffyuv ljpeg r10k; do time ffmpeg -i "$f" -vcodec $vcodec "${f//.mkv/-$vcodec.mkv}"; done

and also manual

ffmpeg -i 2020.03.05.15.18.34_2020.03.05.15.18.43.mkv -c:v libx264 -preset veryslow -crf 0 2020.03.05.15.18.34_2020.03.05.15.18.43-libx264-veryslow-crf0.mkv

and the original one beats them all

$> ls -lS 2020.03.05.15.18.34_2020.03.05.15.18.43* -rw------- 1 yoh yoh 2654234059 Mar 5 16:10 2020.03.05.15.18.34_2020.03.05.15.18.43-r10k.mkv -rw------- 1 yoh yoh 350986442 Mar 5 16:10 2020.03.05.15.18.34_2020.03.05.15.18.43-ffvhuff.mkv -rw------- 1 yoh yoh 350986442 Mar 5 16:10 2020.03.05.15.18.34_2020.03.05.15.18.43-huffyuv.mkv -rw------- 1 yoh yoh 114410113 Mar 5 16:08 2020.03.05.15.18.34_2020.03.05.15.18.43-png.mkv -rw------- 1 yoh yoh 54690284 Mar 5 16:08 2020.03.05.15.18.34_2020.03.05.15.18.43-flashsv.mkv -rw------- 1 yoh yoh 48942534 Mar 5 16:08 2020.03.05.15.18.34_2020.03.05.15.18.43-ffv1.mkv -rw------- 1 yoh yoh 41958745 Mar 5 16:08 2020.03.05.15.18.34_2020.03.05.15.18.43-qtrle.mkv -rw------- 1 yoh yoh 13282631 Mar 5 16:10 2020.03.05.15.18.34_2020.03.05.15.18.43-zmbv.mkv -rw------- 1 yoh yoh 3933457 Mar 5 16:08 2020.03.05.15.18.34_2020.03.05.15.18.43-gif.mkv -rw------- 1 yoh yoh 2227212 Mar 5 16:10 2020.03.05.15.18.34_2020.03.05.15.18.43-libx264-veryslow-crf0.mkv -rw------- 1 yoh yoh 293398 Mar 5 15:54 2020.03.05.15.18.34_2020.03.05.15.18.43.mkv

so the (original) AVC is the unquestionable "winner", but I don't think it was lossless when I look at the video and since explicit attempt to compress lossless using AVC resulted in x10 larger file. So -- we need to check either compression is done in hardware and ffmpeg just dumps it (then there could be some API to change codec/settings) or it compresses on the fly (and thus we would need to adjust ffmpeg invocation)

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/ReproNim/reprostim/issues/4?email_source=notifications&email_token=AA6IHQJYYG46MSO6ZK6ILZDRGAKBNA5CNFSM4K34WZS2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEN66XDY#issuecomment-595454863, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA6IHQO26EVHAU2G2ADNJYDRGAKBNANCNFSM4K34WZSQ .

[yarikoptic]

My understanding is that mpeg-4 (AVC) aks (H.264) is a state of the art lossy codec.

yes -- state of art & no -- it does support lossless compression, that is what I did with -c:v libx264 -preset veryslow -crf 0 where -crf 0 should use lossless, see https://trac.ffmpeg.org/wiki/Encode/H.264

What you appear to have done above is to convert from a lossy recording to some other compression.

yeap

But is we want to do lossless encoding, we need to do the initial encoding with a lossless codec

yeap. that is why I wondered "we need to check either compression is done in hardware and ffmpeg just dumps it (then there could be some API to change codec/settings) or it compresses on the fly (and thus we would need to adjust ffmpeg invocation)". There would be no point to received stream from the device in lossy and save it lossless. So the question remains -- what video stream we (well -- ffmpeg) receives from the device, and either we have an instrumentation (can change settings via magwell API or otherwise) over it?

[andycon]

According to the Magewell website the default format for video capture is YUY2: "Support for capture resolutions up to 2048x2160 Support for capture frame rates up to 120fps (Actual capture frame rate can be limited by the USB bandwidth and internal working frequency. Typical capture frame rates on the Intel USB3.0 controller are as follows.) 1920x1080 YUY2 (up to 75fps) 1920x1080 RGB24 (up to 60fps) Support for YUY2 & UYVY 4:2:2 8-bit Support for RGB24 & RGB32 4:4:4 8-bit The default capture format is YUY2. More capture format can be set using USB Capture Utility."

YUY2 is uncompressed, but I do not know whether also lossless, though.

[yarikoptic]

RGB24 and YUY2 are just color encoding schemes or how to represent colored data in the stream/file... I think card should be doing some compression before feeding data via usb3, probably using some efficient proprietary quantizer in case of lossy compression. I could be wrong (as always). googling up: https://trac.ffmpeg.org/wiki/Capture/Webcam suggests:

List device capabilities
To list available formats (supported pixel formats, video formats, and frame sizes) for a particular input device:

$ ffmpeg -f v4l2 -list_formats all -i /dev/video0
…
[video4linux2,v4l2 @ 0xf07d80] Raw       :   yuyv422 :     YUV 4:2:2 (YUYV) : 640x480 160x120 176x144 320x176 320x240 352x288 432x240 544x288 640x360
[video4linux2,v4l2 @ 0xf07d80] Compressed:     mjpeg :                MJPEG : 640x480 160x120 176x144 320x176 320x240 352x288 432x240 544x288 640x360
Alternatively you could use v4l2-ctl --list-formats-ext to list available formats.

so we should try that... my guess that what we captured wasn't raw but came already compressed. did you get the magwell from the college or is it still in your office?

[andycon]

Magewell is still in my office. I have not been back to campus yet.

On Mon, Mar 30, 2020 at 10:03 PM Yaroslav Halchenko < notifications@github.com> wrote:

RGB24 and YUY2 are just color encoding schemes or how to represent colored data in the stream/file... I think card should be doing some compression before feeding data via usb3, probably using some efficient proprietary quantizer in case of lossy compression. I could be wrong (as always). googling up: https://trac.ffmpeg.org/wiki/Capture/Webcam suggests:

List device capabilities

To list available formats (supported pixel formats, video formats, and frame sizes) for a particular input device:

$ ffmpeg -f v4l2 -list_formats all -i /dev/video0

…

[video4linux2,v4l2 @ 0xf07d80] Raw : yuyv422 : YUV 4:2:2 (YUYV) : 640x480 160x120 176x144 320x176 320x240 352x288 432x240 544x288 640x360

[video4linux2,v4l2 @ 0xf07d80] Compressed: mjpeg : MJPEG : 640x480 160x120 176x144 320x176 320x240 352x288 432x240 544x288 640x360

Alternatively you could use v4l2-ctl --list-formats-ext to list available formats.

so we should try that... my guess that what we captured wasn't raw but came already compressed. did you get the magwell from the college or is it still in your office?

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/ReproNim/reprostim/issues/4#issuecomment-606353078, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA6IHQO7PNP5DDH6XBX2V7LRKFFPJANCNFSM4K34WZSQ .

[yarikoptic]

looking at a sample video of an exciting fixation cross I see quite a bit of compression effects . What was the verdict about ability to capture/record files in "raw" or lossless mode @andycon ?

[yarikoptic]

attn @vmdocua: I think we need to get back to this issue. With current setup, if someone forgets their laptop attached to the stimuli display for a night or weekend, then we get TB sized videos with a static screen image. Filesystem level compression seems to not kick in (only CoW helps with a duplicate) and thus we do consume that much space for no good reason:

root@reproiner:/data# du -sch reprostim/.git/annex/objects/Mq/Px/MD5E-s738339061760--23f4feaf58609aff4a38b43bd6988bb0.13.mkv/MD5E-s738339061760--23f4feaf58609aff4a38b43bd6988bb0.13.mkv 
688G    reprostim/.git/annex/objects/Mq/Px/MD5E-s738339061760--23f4feaf58609aff4a38b43bd6988bb0.13.mkv/MD5E-s738339061760--23f4feaf58609aff4a38b43bd6988bb0.13.mkv
688G    total
root@reproiner:/data# compsize .
Processed 277 files, 56078 regular extents (111712 refs), 253 inline.
Type       Perc     Disk Usage   Uncompressed Referenced  
TOTAL       99%      684G         687G         1.3T       
none       100%      681G         681G         1.3T       
lzo         52%      3.1G         6.0G          12G    

even some trivial compression which does some kind of delta on frames should be of help here...

ATM for some reason ffmpeg reports only raw available, whenever before there was also mjpeg... odd:

reprostim@reproiner:/data/reprostim$ ffmpeg -f v4l2 -list_formats all -i /dev/video0
ffmpeg version 5.1.4-0+deb12u1 Copyright (c) 2000-2023 the FFmpeg developers
...
[video4linux2,v4l2 @ 0x55b85c238e00] Raw       :     yuyv422 :           YUYV 4:2:2 : 640x360 640x480 720x480 720x576 768x576 800x600 856x480 960x540 1024x576 1024x768 1280x720 1280x800 1280x960 1280x1024 1368x768 1440x900 1600x1200 1680x1050 1920x1080 1920x1200 2048x1080
/dev/video0: Immediate exit requested

but given that we just use -c:v copy in our invocation, we might just look into tuning that up to something else to do compress at least somewhat, ideally lossless-ly.

here are ideas chatgpt gave but I have not checked/tried them:

For lossless video compression using FFmpeg, you have a few codec options that will allow you to compress your video without losing any quality. Here are some of the codecs you can use for lossless compression: 1. **H.264 (x264)** - While H.264 is typically used for lossy compression, it also has a lossless mode. You can enable this in FFmpeg with the `-crf 0` option for the libx264 codec. The command might look like this: ```bash ffmpeg -i input.mp4 -c:v libx264 -preset veryslow -crf 0 -c:a copy output.mkv ``` 2. **H.265 (x265)** - Similar to x264, x265 also has a lossless mode that can be enabled with `-x265-params lossless=1`. Here's an example command: ```bash ffmpeg -i input.mp4 -c:v libx265 -x265-params lossless=1 -c:a copy output.mkv ``` 3. **FFV1** - FFV1 is a codec developed within FFmpeg itself. It's often used for archiving and preservation due to its high-quality lossless compression. To use FFV1: ```bash ffmpeg -i input.mp4 -c:v ffv1 -level 3 -g 1 -slices 24 -slicecrc 1 -c:a copy output.mkv ``` 4. **Ut Video** - Ut Video is a lossless video codec that provides good compression and is faster than FFV1 but not as widely supported. To use Ut Video: ```bash ffmpeg -i input.mp4 -c:v utvideo -c:a copy output.mkv ``` 5. **MagicYUV** - A fast, lossless codec, MagicYUV is designed for 4K and higher resolutions. To use MagicYUV: ```bash ffmpeg -i input.mp4 -c:v magicyuv -c:a copy output.mkv ``` 6. **PNG Sequence** - While not a video codec, converting your video to a sequence of PNG images is a method of lossless storage. Each frame is stored as an individual PNG file. This is more commonly used in scenarios where frame-by-frame editing or analysis is needed. ```bash ffmpeg -i input.mp4 -c:v png -c:a copy output_%03d.png ``` Remember that lossless compression typically results in larger file sizes compared to lossy compression, as no data is being thrown away. The exact command you'll use will depend on your specific needs, including compatibility with other software or devices and the balance you want to strike between file size and encoding/decoding speed. Choose the codec that best fits your requirements for the task at hand.

[yarikoptic]

here is a (working without fixups... cool) script from chatgpt to test different codecs

```shell #!/bin/bash # Ensure an input file is provided if [ "$#" -ne 1 ]; then echo "Usage: $0 " exit 1 fi INPUT=$1 # H.264 Lossless ffmpeg -i "$INPUT" -c:v libx264 -preset veryslow -crf 0 -c:a copy "${INPUT%.*}-h264.mkv" # H.265 Lossless ffmpeg -i "$INPUT" -c:v libx265 -x265-params lossless=1 -c:a copy "${INPUT%.*}-h265.mkv" # FFV1 ffmpeg -i "$INPUT" -c:v ffv1 -level 3 -g 1 -slices 24 -slicecrc 1 -c:a copy "${INPUT%.*}-ffv1.mkv" # Ut Video ffmpeg -i "$INPUT" -c:v utvideo -c:a copy "${INPUT%.*}-utvideo.mkv" # MagicYUV ffmpeg -i "$INPUT" -c:v magicyuv -c:a copy "${INPUT%.*}-magicyuv.mkv" echo "All conversions completed!" ```

which gives smth like

❯ du -scm 2024*mkv
741 2024.01.05.14.44.01_.mkv
56  2024.01.06.09.28.23_2024.01.06.10.18.13-ffv1.mkv
38  2024.01.06.09.28.23_2024.01.06.10.18.13-h264.mkv
38  2024.01.06.09.28.23_2024.01.06.10.18.13-h265.mkv
82  2024.01.06.09.28.23_2024.01.06.10.18.13-magicyuv.mkv
280 2024.01.06.09.28.23_2024.01.06.10.18.13.mkv
79  2024.01.06.09.28.23_2024.01.06.10.18.13-utvideo.mkv

but what is interesting, is that a sample file which was already "sorted" (I guess using https://github.com/ReproNim/reprostim/blob/HEAD/repro-vidsort) into /inbox/VIDS/2023/01/30/A005527/2023.01.30.10.37.35_2023.01.30.11.10.56.mkv is actually already compressed too into h264 and recompression just makes it larger:

❯ for f in 2023*.mkv ; do s=$(du -m "$f" | tr '\n'  ' '); echo -n "$s: "; ffprobe -v error -select_streams v:0 -show_entries stream=codec_name -of default=noprint_wrappers=1:nokey=1 $f; done
1922    2023.01.30.10.37.35_2023.01.30.11.10.56-h264.mkv : h264
434 2023.01.30.10.37.35_2023.01.30.11.10.56.mkv : h264

... actually conversion is still ongoing (on my laptop) so not sure WTF is going on ...

meanwhile I checked git history to find that in 646fbbe2f8e07b2ebbc0a4a9c8623ceb26e04a93 (commit bundles two changes so changes to config are not described why) on Jan 5 2023 (collection was terminated shortly after) Andy seems went from -f v4l2 -vcodec libx264 -pix_fmt yuv420p -threads 0 to -c:v copy. So that might explain why compression disabled now but prior images had some enabled... I have tried to get a sample recording and then recompress to see that it seems to record quite efficiently (CPU was at 20%) any recompression is blowing it up

❯ for f in out*.mkv ; do s=$(du -k "$f" | tr '\n'  ' '); echo -n "$s: "; ffprobe -v error -select_streams v:0 -show_entries stream=codec_name -of default=noprint_wrappers=1:nokey=1 $f; done
4828    out-ffv1.mkv : ffv1
516 out-h264.mkv : h264
652 out-h265.mkv : hevc
25576   out-magicyuv.mkv : magicyuv
112 out.mkv : h264
24140   out-utvideo.mkv : utvideo

Assuming that visual artifact I just reported separately about in #67 is unrelated, I will now revert the changes to the config and apply them to the deployment to regain sensible file sizes... those few prior huge ones would be force dropped/removed.

[yarikoptic]

@vmdocua if you could please check if device itself can provide some compression on board, so we could avoid relying on data collection box CPU (seems to be not keeping up) -- would be great.

[yarikoptic]

not really sure why on this device we now not getting Compressed: mjpeg : option for ffmpeg -f v4l2 -list_formats all -i /dev/video0 (or /dev/video2 which is the same 2nd device). For the HDMI one (cheaper newer) which I have at home for development I do see mjpeg option which is a compression but I guess would be lossy... :-/

❯ sudo ffmpeg -f v4l2 -list_formats all -i /dev/video0
ffmpeg version 6.1.1-1 Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 13 (Debian 13.2.0-9)
  configuration: --prefix=/usr --extra-version=1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libharfbuzz --enable-libfribidi --enable-libglslang --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librist --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --disable-sndio --enable-libjxl --enable-pocketsphinx --enable-librsvg --enable-libvpl --disable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libsvtav1 --enable-libx264 --enable-libplacebo --enable-librav1e --enable-shared
  libavutil      58. 29.100 / 58. 29.100
  libavcodec     60. 31.102 / 60. 31.102
  libavformat    60. 16.100 / 60. 16.100
  libavdevice    60.  3.100 / 60.  3.100
  libavfilter     9. 12.100 /  9. 12.100
  libswscale      7.  5.100 /  7.  5.100
  libswresample   4. 12.100 /  4. 12.100
  libpostproc    57.  3.100 / 57.  3.100
[video4linux2,v4l2 @ 0x5627280f7000] Compressed:       mjpeg :          Motion-JPEG : 1920x1080 320x180 320x240 352x288 424x240 640x360 640x480 848x480 960x540 1280x720
[video4linux2,v4l2 @ 0x5627280f7000] Raw       :     yuyv422 :           YUYV 4:2:2 : 640x480 320x180 320x240 352x288 424x240 640x360 848x480 960x540 1280x720 1920x1080
[in#0 @ 0x5627280f6e40] Error opening input: Immediate exit requested
Error opening input file /dev/video0.