search

shimat / opencvsharp

OpenCV wrapper for .NET
Apache License 2.0
5.43k stars 1.15k forks source link

How to Encode and Output VideoWriter with High Quality #1696

Open GodOMG opened 3 months ago

GodOMG commented 3 months ago

Through testing, I found that no matter how I configured VideoWriter, when recording webCam at 640×480 resolution and 15FPS, the minimum bitrate was over 1000. And the FFMPEG backend does not support H264. MSMF backend H264 bit rate 4000+.

Environment

win 10 64bit .NET 8 opencvsharp 4.10.0.20240616 opencvsharp.Extensions 4.10.0.20240616 opencvsharp.runtime.win 4.10.0.20240616

MP4V:

VideoWriter _videoWriter = new VideoWriter("output.m4v", VideoCaptureAPIs.FFMPEG, FourCC.MP4V, 15, new OpenCvSharp.Size(640, 480));

The File MediaInfo:
General
Format                                   : MPEG-4
Codec ID                                 : M4V  (M4V /isom/iso2)
File size                                : 1.66 MiB
Duration                                 : 9 s 67 ms
Overall bit rate mode                    : Variable
Overall bit rate                         : 1 540 kb/s
Frame rate                               : 15.000 FPS
Writing application                      : Lavf58.76.100

Video
ID                                       : 1
Format                                   : MPEG-4 Visual
Format profile                           : Simple@L1
Format settings, BVOP                    : No
Format settings, QPel                    : No
Format settings, GMC                     : No warppoints
Format settings, Matrix                  : Default (H.263)
Codec ID                                 : mp4v-20
Duration                                 : 9 s 67 ms
Bit rate mode                            : Variable
Bit rate                                 : 1 539 kb/s
Maximum bit rate                         : 9 216 kb/s
Width                                    : 640 pixels
Height                                   : 480 pixels
Display aspect ratio                     : 4:3
Frame rate mode                          : Constant
Frame rate                               : 15.000 FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Compression mode                         : Lossy
Bits/(Pixel*Frame)                       : 0.334
Stream size                              : 1.66 MiB (100%)
Writing library                          : Lavc58.134.100

H264

VideoWriter _videoWriter = new VideoWriter("output.mp4", VideoCaptureAPIs.MSMF, FourCC.H264, 15, new OpenCvSharp.Size(640, 480));

The File MediaInfo:
General
Format                                   : MPEG-4
Format profile                           : Base Media / Version 2
Codec ID                                 : mp42 (mp41/isom)
File size                                : 4.69 MiB
Duration                                 : 8 s 533 ms
Overall bit rate                         : 4 608 kb/s
Frame rate                               : 15.000 FPS
Encoded date                             : 2024-08-18 21:38:40 UTC
Tagged date                              : 2024-08-18 21:38:40 UTC

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : Constrained Baseline@L2.2
Format settings                          : 1 Ref Frames
Format settings, CABAC                   : No
Format settings, Reference frames        : 1 frame
Format settings, GOP                     : M=1, N=30
Codec ID                                 : avc1
Codec ID/Info                            : Advanced Video Coding
Duration                                 : 8 s 533 ms
Bit rate                                 : 4 607 kb/s
Width                                    : 640 pixels
Height                                   : 480 pixels
Display aspect ratio                     : 4:3
Frame rate mode                          : Constant
Frame rate                               : 15.000 FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 1.000
Stream size                              : 4.69 MiB (100%)
Encoded date                             : 2024-08-18 21:38:40 UTC
Tagged date                              : 2024-08-18 21:38:40 UTC
mdhd_Duration                            : 8533
Codec configuration box                  : avcC

FFMPEG CLI Recording

ffmpeg -f dshow -i video="XHWS-179-AF1" -vf "vflip,scale=640:-1,format=yuv420p" -r:v 15 -c:v libx264 output.mp4

The File MediaInfo:
General
Format                                   : MPEG-4
Format profile                           : Base Media
Codec ID                                 : isom (isom/iso2/avc1/mp41)
File size                                : 407 KiB
Duration                                 : 13 s 934 ms
Overall bit rate                         : 239 kb/s
Frame rate                               : 15.000 FPS
Writing application                      : Lavf61.5.101

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High@L2.2
Format settings                          : CABAC / 4 Ref Frames
Format settings, CABAC                   : Yes
Format settings, Reference frames        : 4 frames
Codec ID                                 : avc1
Codec ID/Info                            : Advanced Video Coding
Duration                                 : 13 s 934 ms
Bit rate                                 : 237 kb/s
Width                                    : 640 pixels
Height                                   : 480 pixels
Display aspect ratio                     : 4:3
Frame rate mode                          : Constant
Frame rate                               : 15.000 FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 0.051
Stream size                              : 403 KiB (99%)
Writing library                          : x264 core 164
Encoding settings                        : cabac=1 / ref=3 / deblock=1:0:0 / analyse=0x3:0x113 / me=hex / subme=7 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=6 / lookahead_threads=1 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=15 / scenecut=40 / intra_refresh=0 / rc_lookahead=40 / rc=crf / mbtree=1 / crf=23.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00
Color range                              : Limited
Color primaries                          : BT.709
Codec configuration box                  : avcC

I hope to record high-quality video files in real time like FFMPEG CLI records webCam (good compression and clear image quality)?

Is it that the file opencv_videoio_ffmpeg4100.dll does not have libx264 integrated? Do I need to manually compile opencv and include libx264 or Cisco's openh264 when integrating ffmpeg to make the ffmpeg backend support H264?

GodOMG commented 3 months ago

In the underlying opencv 4.10.0 source code, I found the Cisco openh264 version 1.8 that corresponds to this version.

The latest version of openh264 that I downloaded before is not supported.

By copying the 1.8 version of openh264.dll to system32 or the current project's exe generation directory, I can use h264 encoding when I specify the use of ffmpeg backend recording, but the compression effect of the recording quality is still not very good.