search

voc / voctomix

Full-HD Software Live-Video-Mixer in python
https://c3voc.de/
MIT License
578 stars 107 forks source link

bring back the TCP source reconnect #244

Open fightling opened 5 years ago

fightling commented 5 years ago

I think I've made a mistake when I threw away the TCPSingleConnection class which offers a separate gst pipeline for every TCP connection. It is implementing a gst.fdsrc which needs a file descriptor within the inital pipeline definition.

@MaZderMind Is this maybe the reason why the inter* thing was started?

I thought I later would be able to force gstreamer to reconnect the tcpserversrc. But until now, I'm without success.

So because @danimo seems to got to a problem about pause loops that might be related to this fault of mine.

Any ideas? Is my interpretation correct?

MaZderMind commented 5 years ago

@MaZderMind Is this maybe the reason why the inter* thing was started?

Yes. That was the exact point (quite early in development) where I introduced these elements to be able to restart individual sources upon de/reconnection. At this time we hat a system where all sources and sinks were individual processes. This has changed since, as we're using the decklink*src-elements as primary inputs.

So because @danimo seems to got to a problem about pause loops that might be related to this fault of mine.

There are multiple ways to fix this. One obvious would be to move the playback-logic for the Pause-Loop into voc2mix. A GStreamer-based Loop-Source could be seen here: https://github.com/voc/voctomix/blob/master/example-scripts/gstreamer/source-background-loop.py Although TCPSources give us a lot flexibility in trying new Sources, I don't think they are strictly required for a production use.

I thought I later would be able to force gstreamer to reconnect the tcpserversrc. But until now, I'm without success.

It should be possible to stop and restart individual elements, but they might emit EOS-Elements down the pipeline which can cause havoc in compositors and other elements down the stream. gstreamer-devel@lists.freedesktop.org seems to be quite helpful to solve these kind of problems.

dnet commented 5 years ago

Although TCPSources give us a lot flexibility in trying new Sources, I don't think they are strictly required for a production use.

We use TCP sources in production for two purposes:

How hard would it be to perform these in any alternate manner if support for TCP sources gets dropped?

derpeter commented 4 years ago

As far as i understand the concept of the TCP sources in gstreamer they are not meant to be reused. Instead we could go for an UDP source https://gstreamer.freedesktop.org/documentation/udp/udpsrc.html?gi-language=c#udpsrc-page and depending on the use case we can put a e.g. rtp decoder behind hit or what ever level of stream control we need for the application.

Another option to archive the old behavior would be to use an inter element like in the past. As we are here on the edge of the pipline we just need a element behind the interelemet which can deal with changing inputs.

markvdb commented 4 years ago

/me silently hopes the FOSDEM use case with mpeg ts input over tcp will still be supported in vox2mix......

Op wo 15 jan. 2020 om 20:50 schreef derpeter notifications@github.com:

As far as i understand the concept of the TCP sources in gstreamer they are not meant to be reused. Instead we could go for an UDP source https://gstreamer.freedesktop.org/documentation/udp/udpsrc.html?gi-language=c#udpsrc-page and depending on the use case we can put a e.g. rtp decoder behind hit or what ever level of stream control we need for the application.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/voc/voctomix/issues/244?email_source=notifications&email_token=AAF6ECEEERVUQIVESVHB4HDQ55SJRA5CNFSM4GYEVEJ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEJBS4OQ#issuecomment-574828090, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAF6ECANSYZ3E7ZP5OQFHZLQ55SJRANCNFSM4GYEVEJQ .

-- Mark Van den Borre Hogestraat 16 3000 Leuven, België +32 486 961726

danimo commented 4 years ago

/me silently hopes the FOSDEM use case with mpeg ts input over tcp will still be supported in vox2mix......

Which sources are you running via TCP? Can you modify the source to send the contents via RTP instead (e.g. ffmpeg)?

derpeter commented 4 years ago

@markvdb as long as the source is not disconnecting this still works as before. But the current implementation can't handle reconnects.

As you get the mpeg ts on bananapi it should make not much difference if its just wrapped in TCP or muxed into something which would allow reconnect handling.

I'm currently not sure what would be the best protocol to do this. As fosdem is close i would suggest to start working on this soon.

@danimo have a look at https://github.com/FOSDEM/video/tree/master/hardware for details

markvdb commented 4 years ago

No worries about time pressure. Still using voctomix 1 for the current edition.

I just mentioned our use case as we seem to be one of the biggest users of the tcp sources feature after moving away from udp multicast ingestion for network reliability reasons...

Op do 16 jan. 2020 om 08:44 schreef derpeter notifications@github.com:

@markvdb as long as the source is not disconnecting this still works as before. But the current implementation can't handle reconnects.

As you get the mpeg ts on bananapi it should make not much difference if its just wrapped in TCP or muxed into something which would allow reconnect handling.

I'm currently not sure what would be the best protocol to do this. As fosdem is close i would suggest to start working on this soon.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

krokodilerian commented 4 years ago

Just a clarification, for FOSDEM our transport is TCP, but we ingest this on the node that runs voctomix, and we can push this via whatever-available, via loopback, being UDP, RTP, etc. The problem with those is that they don't handle drops very well and if my source reconnects, i'll have a gap in the UDP stream. Having always-up not-reconnecting-ever sources is pretty much impossible as we're doing everything over the network.

So the question is "what's the best way to send video streams via the loopback to voc2mix that can live with data being missing for a while" (and is a question for 2021 :) )

derpeter commented 4 years ago

well thats what we discussed above. It doesn't make a difference if you do it via loopback or over the network at least not from voc2mix point of view. Of course the probability of packet loss is higher on the network but in both cases you would need to restart the core to recover.

What you could do is receive the stream from you box with e.g. an gstreamer pipline that can provides a black image torwards voctomix if no signal is comming in. If you assume that this component never restarts you could keep using the tcp port.

As we are talking about 2021 we hopefully have by that time a better solution implemented, as said above probably something like rtp oder rtmp.

Than you can just tell you banapi to send rtp/rtmp (or what ever we turns out to be the best option) directly to vocotocore.

I'm currently experimenting with an Pi4 and a USB3 elgato cam link capture device which i want to use for a similar use case like you do.

markvdb commented 3 years ago

Just fyi, we used voctomix 1.x at FOSDEM 2021. The builtin tcp reconnect was incredibly helpful again. Two types of sources this time:

Will definitely look into your suggestion of putting a proxy that accepts reconnects in front of voctomix 2 for next time though. Unless there has been any movement embedding the reconnection functionality into voctomix 2 core?

derpeter commented 3 years ago

hi @markvdb in 2020 sadly less development happend then expected. Most of it was related to https://c3voc.de/wiki/projects:isdn and porting back that features to voxtomix2. I hope to finish that backporting soon so we can have a new release. My current view on the tcp connection issue is that i would rather add SRT as a new input (its currently only supported as output) than making tcp work. SRT brings the big advantage of a context, so the srt sink is aware thats happening on source side and can e.g. handle reconnects. But if tcp is still necessary i think we can have that to by adding an https://gstreamer.freedesktop.org/documentation/inter/index.html between the tcp input and the core pipeline. But i can't give an ETA :-(.

markvdb commented 3 years ago

Hi @derpeter ! Whatever standards based form of network based voctomix input able to handle reconnects should probably work for us. SRT input actually looks like it might even bring us some extra advantages on top for almost free.

We'd have to play with it and document things a bit first obviously first. Anywhere in particular for us to keep track of the SRT input functionality landing in voctomix2?

derpeter commented 3 years ago

Let us know when you have something in writing about the fossdem setup we are as usual very interested. As we could not use voctomix for the ccc rc3 event i'm very happy that it was used again by you guys.

markvdb commented 3 years ago

I hope I'm not abusing this issue too much. If so, please don't hesitate to move this info somewhere more suitable.

This was a rock stable spit and duct tape solution really:

derpeter commented 3 years ago

not at all, very intersting that you used vocotomix for live stream inputs. Use case input is very appriciated!

SonnyWalkman commented 3 years ago

Have you tried RidgeRun interpipes in voctomix core? It offer less queues and switches between multiple sources? There was some mention about using them a few years ago. I love how voctomix is modular and easy enough to add to. I'm using parts of voctomix for project I'm working on. A digital amateur TV repeater. Basically a switcher with dvb compliant decoding switching with a muxer connected at the mix point to encode to TS stream before a transmitter. There has been great work put into the code base. Respe t the clarity in the source. Few more comment be a bonus.

derpeter commented 3 years ago

maybe @MaZderMind played with them. The last two years where more in the directeion of one big pipline to get audio video sync issue fixed (which works so far). But i have the feeling that splitting it up again on the "edges" as in on the inputs/outputs will be the next step. So interpipes are interesting for that.