search

centricular / gstcefsrc

A simple gstreamer wrapper around Chromium Embedded Framework
83 stars 45 forks source link

Deadlock when mixing Audio and Video #69

Closed SteveMcFarlin closed 1 year ago

SteveMcFarlin commented 1 year ago

My issue is very similar to https://github.com/centricular/gstcefsrc/issues/59. The difference is my state flow is different.

gstreamer 1.20.3 gstcefsrc master with this patch applied: https://github.com/philn/gstcefsrc/commit/3b9d7eb6f5a576ae948cff92175c2f0ac2c1863d

My coded pipeline is identical to the one in the README. I enter into PAUSED to preload a web page. After it has loaded I enter into PLAYING. This will cause a deadlock when trying to send an EOS on the message bus. If I remove the audio then everything works as expected. Also replacing gstcefsrc/demux with a simple audio and video test source works fine. I did try setting queues to leaky, but this ends up causing AV sync issues.

My only thoughts are to maybe dynamically configure the output to the pipeline prior to going into PLAYING state, or modifying gstcefsrc to monitor the states and simply dump all AV data unless in PLAYING state. I really need the page to be preloaded and data flowing but not recording while in the PAUSED state. Maybe using a splitmuxsink might work. I am totally open to any ideas anyone may have.

SteveMcFarlin commented 1 year ago

Adding a state transition monitor to cefsrc element class, and then adding a simple toggle to dump data unless in the PLAYING state fixes my issue.

fairbairn commented 1 year ago

We've encountered the same issues with deadlocks.

On the latest master branch here, if the CPU is under heavy load, the cefsrc / cefdemux flow deadlocks and will not recover. The pipeline stalls and won't proceed.

We had hoped that phil's patch might work, but honestly, we're uncertain how to perform the patch to this code base from his. We attempted even to run phil's audio-ts branch latest as the plugin instead, but it's out of sync with certain features in the latest CEF, so we reverted. We used the 3 modified files from audio-ts on top of this master, and that ran, but it hung.

@philn any chance the "hack" above could be integrated into your solution? Obviously filling up buffers, detecting them, dumping the audio, and changing the play state is non-ideal, because the audio is going to get corrupted, but it would at least continue.

MathieuDuponchelle commented 1 year ago

@philn 's branch rebased with no conflicts on master, I have merged it now. You probably want to give master a spin again. As for this issue here, a test case with that two-step approach to going to PLAYING, showing the deadlock reliably would be helpful :)

fairbairn commented 1 year ago

That's great.

I do want to add, that from master, we altered the pipeline to instead sink with an flvmux and route the output to an NGINX RTMP server, and it never hung. We overtaxed the CPU, ran 12 pipelines in parallel on a GPU machine, and it never hung.

So I believe the problems stated before, although certainly a concern in terms of stability, may have been caused by the mux's characteristics.

MathieuDuponchelle commented 1 year ago

Perhaps simply a matter of porting matroskamux from collectpads to aggregator then

fairbairn commented 1 year ago

Yes. In fact, using a different mux seems to resolve the issue.

On Mon, May 1, 2023, 9:24 AM Mathieu Duponchelle @.***> wrote:

Perhaps simply a matter of porting matroskamux to collectpads then

— Reply to this email directly, view it on GitHub https://github.com/centricular/gstcefsrc/issues/69#issuecomment-1529709530, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABILNGO3XOLR4UP4N4OG6B3XD62R3ANCNFSM6AAAAAAUETP5D4 . You are receiving this because you commented.Message ID: @.***>