search

raspberrypi / userland

Source code for ARM side libraries for interfacing to Raspberry Pi GPU.
BSD 3-Clause "New" or "Revised" License
2.05k stars 1.09k forks source link

Level 4.2 encoding #678

Closed Consti10 closed 3 years ago

Consti10 commented 3 years ago

Describe the bug Level 4.2 encoding increases encoding latency when comparing 720p60 and 720p120, even though this is contradictionary - with a higher framerate, any frames the encoder holds on to should take less time to make it through the pipeline.

To reproduce Compare the encoding time when running raspivid at 720p60 vs 720p120 , by either measuring the end-to-end latency or measuring the encoding time like this: https://github.com/raspberrypi/userland/compare/master...Consti10:master#diff-78efcafc8e61849b1673d77a45597d493622cdc897507cd1f955adeada467c4c

Expected behaviour Higher fps should decrease encoding time (frame drops aside)

Actual behaviour Higher fps actually increase encoding time (frame drops aside)

System Rpi 3B+, latest raspbian fw

pelwell commented 3 years ago

I don't follow your logic - higher frame rate means more frames means more work. You might want them to be processed quicker but I don't see how that can be the case.

6by9 commented 3 years ago

The H264 block is specified as level 4.0. It allows the configuration to be set to level 4.2 predominantly to allow for transcoding to that level (not real time).

Raspivid requests 3 + vcos_max(0, (state->framerate-30)/10) frames on the output of the ISP which are also passed to the input of the video encoder. https://github.com/raspberrypi/userland/blob/master/host_applications/linux/apps/raspicam/RaspiVid.c#L1584 What you are seeing is that that pool of images is exhausted because the encoder can't keep up.

Consti10 commented 3 years ago

I'm gonna close the issue and come back after I got more testing.