expose H264 encoder settings

[Zireael]

Hi, Is it possible to control the quality of captured videos through settings H264 encoder parameters? I mean, would you be able to add a tab with custom settings where we could choose compression level/bitrate/resolution/framerate/smoothing/sharpening for both videos saved to memory card and streamed? Maybe include several different profile settings to choose from and let user save custom profiles.

Reason I'm asking because the video produced is more pixelated and sharpened than I'd expect. Not sure if it's hardware or software (encoder) related. I see in source files two resolution values hardcoded (360 and 1080), but can't discern any other settings used for the encoder.

Thanks in advance!

[roleoroleo]

The encoding process is made by the kernel driver. The proc file system exposes a lot of file with a lot of settings but I don't have documentation to describe it. Try to navigate /proc/mstar

Furthermore there are a lot of parameters that you can configure with setprop:

/home/yi-hack/bin # getprop
[Video.0.H264VBR.ChangePos]: [100]
[Video.0.H264VBR.Gop]: [40]
[Video.0.H264VBR.IPQPDelta]: [-2]
[Video.0.H264VBR.MaxBitrate]: [1126400]
[Video.0.H264VBR.MaxIQp]: [45]
[Video.0.H264VBR.MaxPQp]: [45]
[Video.0.H264VBR.MinIQp]: [25]
[Video.0.H264VBR.MinPQp]: [25]
[Video.0.H264VBR.StatTime]: [-1225359984]
[Video.0.RC.RcVersion]: [0]
[Video.0.RC.RowQpDelta]: [0]
[Video.0.RC.method]: [H264_VBR]
[Video.1.H264VBR.ChangePos]: [100]
[Video.1.H264VBR.Gop]: [40]
[Video.1.H264VBR.IPQPDelta]: [-2]
[Video.1.H264VBR.MaxBitrate]: [358400]
[Video.1.H264VBR.MaxIQp]: [45]
[Video.1.H264VBR.MaxPQp]: [45]
[Video.1.H264VBR.MinIQp]: [20]
[Video.1.H264VBR.MinPQp]: [20]
[Video.1.H264VBR.StatTime]: [-1225359984]
[Video.1.RC.RcVersion]: [0]
[Video.1.RC.RowQpDelta]: [0]
[Video.1.RC.method]: [H264_VBR]
[init.svc.rcs]: [stopped]
[init.svc.ueventd]: [running]
[mi.debug.proc.ADEC.ready]: [1]
[mi.debug.proc.AENC.ready]: [1]
[mi.debug.proc.AI.ready]: [1]
[mi.debug.proc.AO.ready]: [1]
[mi.debug.proc.MI.ready]: [1]
[mi.debug.proc.OSD.ready]: [1]
[mi.debug.proc.VENC.ready]: [1]
[mi.debug.proc.VI.ready]: [1]
[mi.osd.gop.use]: [0]
[mi.sys.shrink_mem]: [1]
[mi.venc.bufcnt]: [1]
[mi.venc.stop.flush]: [1]
[mi.venc.sub.bufcnt]: [1]
[mi.vi.bufcnt]: [3]
[mi.vi.img.sub]: [0]
[mi.vi.src]: [2]
[mi.vi.sub.bufcnt]: [3]
[mi.vi.sub.height]: [360]
[mi.vi.sub.width]: [640]
[mi.vi.yuvmon.pat]: [/home/ms]
[mi.video.height.force.aligned32]: [1]
[mstar.omx.avqe.aecgain]: [2]
[mstar.omx.avqe.aecmode]: [15]
[mstar.omx.gop.disable]: [1]
[mstar.omx.log.level]: [1]
[persist.sys.usb.config]: [adb]
[ro.allow.mock.location]: [0]
[ro.baseband]: [unknown]
[ro.boot.console]: [ttyS0]
[ro.bootloader]: [unknown]
[ro.bootmode]: [unknown]
[ro.debuggable]: [1]
[ro.factorytest]: [0]
[ro.hardware]: [mstarinfinity3(flatteneddevicet]
[ro.revision]: [0]
[ro.secure]: [1]
[ro.serialno]: []
[ro.system_rcs]: [1]

However I have not done any tests on this and my knowledge of video algorithms is not enough If someone wants to do some tests...

[Zireael]

Thanks! This is good start! The most interesting out of these parameters would be [Video.0.H264VBR.MaxBitrate]: [1126400] => 140.8 kB [Video.1.H264VBR.MaxBitrate]: [358400] => 44.8kB [mi.vi.sub.height]: [360] [mi.vi.sub.width]: [640]

Upping max bitrate of the of the stream should improve the quality of the encoded video, at the expense of bigger file size. The current max size for HD (140.8 kB) roughly matches what I've seen in camera output stream. One worrying thing is the mi.vi.sub.width / height being 640/360. Hard to say, but is is the actual size of the video frame and it gets upscaled to 1920x1080?

google-fu here (https://blog.csdn.net/fc34235/article/details/79992789) tells me that

are some kind of pixel quantization matrix sizes. Halving or doubling them might show some difference in picture quality.

The [Video.0.H264VBR.Gop]: [40] is for setting size of keyframes. At 20 fps and gop 40 frames, you can forward/backward 'seek' video avery 2 seconds (every 2*20 frames).

I'll try playing with ssh in the coming days. I take it I can edit the files in the camera and new settings would take effect after restarting the camera?

[roleoroleo]

I think we are able to change width and height only for the sub stream so only mi.vi.sub. parameters are exposed. Probably mi.vi.main. are fixed 1920x1080.

If you want to modify these settings use /home/app/init.sh where setprop is already called. And restart the camera. If something brick your camera you can reload another firmware and restore the situation with no risks.

Please note that if you change width and height of the sub stream, h264grabber will stop working. And imggrabber too. This change would require some minor tweaks to the software to work.

[Zireael]

I upped the max bitrate 40x and quant parameters 2x with setprop in /home/app/init.sh, but it had no effect

setprop Video.0.H264VBR.MaxBitrate 45056000
setprop Video.1.H264VBR.MaxBitrate 14336000

setprop Video.0.H264VBR.MaxIQp 90
setprop Video.0.H264VBR.MaxPQp 90
setprop Video.0.H264VBR.MinIQp 50
setprop Video.0.H264VBR.MinPQp 50

The HD bitrate still doesn't break 140 KB/s and I don't see any difference in output video (as pixelated as it was) :/

Actually, in the videos saved to the memory card I can see that the video contains two video file streams. HD and SD. A bit of waste of space to save SD stream. Unless they do it to allow you to watch the SD stream on your phone remotely.

General Complete name : ...Video\39M00S60.mp4 Format : MPEG-4 Format profile : Base Media Codec ID : isom (isom/iso2/avc1/mp41) File size : 10.9 MiB Duration : 1 min 0 s Overall bit rate : 1 529 kb/s Encoded date : UTC 2020-01-20 21:39:00 Tagged date : UTC 2020-01-20 21:39:00

Video #1 ID : 1 Format : AVC Format/Info : Advanced Video Codec Format profile : Main@L4 Format settings : CABAC / 1 Ref Frames Format settings, CABAC : Yes Format settings, ReFrames : 1 frame Format settings, GOP : M=1, N=40 Codec ID : avc1 Codec ID/Info : Advanced Video Coding Duration : 1 min 0 s Bit rate : 1 135 kb/s Width : 1 920 pixels Height : 1 080 pixels Original height : 1 088 pixels Display aspect ratio : 16:9 Frame rate mode : Constant Frame rate : 20.000 FPS Color space : YUV Chroma subsampling : 4:2:0 Bit depth : 8 bits Scan type : Progressive Bits/(Pixel*Frame) : 0.027 Stream size : 8.12 MiB (74%) Encoded date : UTC 2020-01-20 21:39:00 Tagged date : UTC 2020-01-20 21:39:00 Color range : Full

Video #2 ID : 2 Format : AVC Format/Info : Advanced Video Codec Format profile : Main@L3 Format settings : CABAC / 1 Ref Frames Format settings, CABAC : Yes Format settings, ReFrames : 1 frame Format settings, GOP : M=1, N=40 Codec ID : avc1 Codec ID/Info : Advanced Video Coding Duration : 1 min 0 s Bit rate : 367 kb/s Width : 640 pixels Height : 360 pixels Display aspect ratio : 16:9 Frame rate mode : Constant Frame rate : 20.000 FPS Color space : YUV Chroma subsampling : 4:2:0 Bit depth : 8 bits Scan type : Progressive Bits/(Pixel*Frame) : 0.080 Stream size : 2.62 MiB (24%) Encoded date : UTC 2020-01-20 21:39:00 Tagged date : UTC 2020-01-20 21:39:00 Color range : Full

Audio ID : 3 Format : AAC Format/Info : Advanced Audio Codec Format profile : LC Codec ID : mp4a-40-2 Duration : 59 s 968 ms Bit rate mode : Constant Bit rate : 23.1 kb/s Channel(s) : 1 channel Channel positions : Front: C Sampling rate : 16.0 kHz Frame rate : 15.625 FPS (1024 SPF) Compression mode : Lossy Stream size : 169 KiB (2%) Default : Yes Alternate group : 1 Encoded date : UTC 2020-01-20 21:39:00 Tagged date : UTC 2020-01-20 21:39:00

[roleoroleo]

I did some tests too. If I change Video.0.H264VBR.MaxBitrate nothing happens: when I request the value with geprop it always gives me the default.

If I change mi.vi.sub.width (or height) instead, the cam goes into infinite reboot loop.

[Zireael]

I'm not that versed in Linux or hardrive drivers (got my cam few days ago and I like tinkering with my stuff, eh). Are the props like Video.0.H264VBR.MaxBitrate hardcoded in YI's compiled software/driver and you modify them using something like get/set API? I'm trying to figure out whether they are saved/exposed in some .ini file and it's a matter of finding the right one.

I've done some playing with hexeditor in the past so if I knew which file holds them, even if it is binary, I could try editing them at source. I don't like that the software is so gimped. They probably limited the bitrate to have to store less on their servers in the cloud if you buy a subscription. Or even so that people don't complain that the memory card holds only like half a day of vids or so...

If you could show me a target file/folder, I'll have a crack at it!

Also, in the /home/app/init.sh at the top I saw something like a storage declaration of 32GB... It says everywhere that the webcam supports max 32GB memory cards, but when I've put in my 256GB card, the YI app showed it's size correctly... Is there some limiter in the software again that caps the video archive to #@GB, or if I put in a 256GB card, will it fill it up with videos entirely?

[roleoroleo]

Too much time since the last post. Open a new issue if necessary.