search

daniilidis-group / ffmpeg_image_transport

image transport that uses libavcodec for compression
Apache License 2.0
43 stars 13 forks source link

How can I speed up encoding+decoding processes? #10

Closed Jihwan-Seo closed 1 year ago

Jihwan-Seo commented 1 year ago

Hi, I'm trying to do video streaming via wireless channel using ROS. Since the wireless channel has low bandwidth, video encoding(ENC) and decoding(DEC) processes are essential, and this code is very useful for what I'm trying to do.

By following the provided instructions, I successfully displayed the video stream after ENC and DEC processes. The video source used for testing was 720p@20Hz, which is not quite high quality. In my case, however, the decoded video has a long delay (about 2.5 seconds). Since the ENC and DEC are performed on the same computer, it is not caused by network delay.

Is there any way to shorten the ENC+DEC time? Currently I am using libx264 codec. (I did not test yet with nvenc due to NVIDIA driver version problem) Can nvenc speed up the ENC+DEC process significantly?

Thank you,

Seo

berndpfrommer commented 1 year ago

A delay of 2s seems like an eternity, but then I never looked at the encoding lag because I was just storing for offline use. How consistent is the lag? If it fluctuates a lot, then most likely it's CPU load. Also check with "top" to see how loaded the machine is. If it's not CPU load then look into the codec options. Is there there some reason why the encoder would delay releasing the encoded packets? I'm not familiar enough with the specifics (it's been a while that I used this code base).

As far as nvenc is concerned: all the hardware based solutions (including nvenc) are blazingly fast. For my project I was encoding 10 streams each 1920x1200@40hz. Impossible (on a 32 core server!) without nvenc, but no problem with nvenc. I needed a high-end card because the non-professional ones can only encode 2 streams at a time. But pretty much any hardware based solution, even with a low end card, should blow your socks off.

berndpfrommer commented 1 year ago

As I found out during the last few days, this is most likely an issue with the buffering. You can reduce the size of the lag by decreasing the "gop" (there is a parameter for that). There are also encoder profiles (like "ll" for low latency that will help).

Jihwan-Seo commented 1 year ago

Sorry for checking and replying to your response too late, and thank you so much for trying to solve my issue even after several months.

I will try to address the problem based on your comments.

Thank you.