search

jc-kynesim / rpi-ffmpeg

FFmpeg work for RPI
Other
111 stars 27 forks source link

Decoding H265 livestream using FFmpeg - Warped, green frames #95

Closed connorh315 closed 3 weeks ago

connorh315 commented 3 weeks ago

I've been using FFmpeg in a "livestream receiver" project in order to decode the frames received. I then output the frames to an OpenGL window. Unfortunately, the frames that are received end up saturated in green. This is going to be a bit of a poorly written issue, but due to the mangled-mess I have made of the code I can't really post the implementation here and I feel like it would take me to longer to make it clean enough to post here then it would to actually just trial-and-error fix it (bad mentality obviously).

Would anyone happen to know what exactly could be causing this mess. As said the structure is HTTP server receives frame from a client, the packets are then passed to FFmpeg on a separate thread to be decoded, the decoded frames are then passed to another frame to be rendered onto a GL window.

Below is a representation of the issue.

image

Even if it's just a push in the right direction of what I should be giving my attention to (i.e. "it's probably something to do with the pixel format being misinterpreted by OpenGL") - It would be much appreciated.

This issue happened when I both compiled FFmpeg new and also used the binaries that can be grabbed from apt.

jc-kynesim commented 3 weeks ago

As you say - pixel format. Green is what you get if all your chroma (UV in YUV) is zero. Y - stride is probably good given you can see anything. Are you sure you are only returning buffers to ffmpeg after they have been displayed? GL needs them to be stable for a period after that (you are going to need to read up on GL for a better answer than that - its too long). If you want an example my hello_wayland project contains what you need (there is a EGL output path hidden in init_window) - though these days it isn't very simple code.

connorh315 commented 3 weeks ago

Perfect, thank you - I am going to start from the ground up again and keep this in mind.

popcornmix commented 3 weeks ago

While I don't know what the correct image should look like, the blocky/smeared artifacts I see look to me like errors/losses in the encoded bitstream buffers, or loss of a reference frame, rather than an incorrect pixel format.

connorh315 commented 3 weeks ago

Thanks for the help guys - removing AVFMT_FLAG_NOBUFFER seems to have fixed the issue. Additionally I believe that the artifacts are because I rushed the code and missed some frees so the memory spikes before it's exhausted and then crashes. When the application first starts the artifacts aren't there (just in case anybody finds this in the future and it's helpful).