steveseguin / vdo.ninja

VDO.Ninja is a powerful tool that lets you bring remote video feeds into OBS or other studio software via WebRTC.
https://vdo.ninja
Other
2.85k stars 810 forks source link

Feature Request: Video Delay on Sender-Side (similar to &micdelay) #943

Open rse opened 2 years ago

rse commented 2 years ago

VDO.Ninja has a &micdelay function which can be used on the sender-side in case the audio is faster then the video (the usual case). I've the situation that my video is faster than my audio, because I've a rather elaborated Cantabile-based audio processing graph and hence I would need a &videodelay option on the sender-side. I know that with &sync it seems that one can achieve a video delay on the receiver side AFAIK, but this I cannot easily use (because for lip-sync at both the room attendees and the recording in the video mixer I want that the sender-side is already correct). The question is whether a sernder-side video delay can be already achieved in VDO.Ninja and if not, perhaps a &videodelay option could be implemented? (PS: as a work-around I'm feeding my camera through ManyCam before VDO.Ninja as it allows to apply a video delay, but a direct option within VDO.Ninja would simplify such a setup, of course).

rse commented 2 years ago

Just one more notice: if applying a millisecond-based delay for video might be too complicated to implement, most of the time it is also sufficient to have a frame-based delay for video. This way for p30 the smallest delay unit is just about 33ms and for p60 it is about 17ms, but even this larger units usually let one avoid extra tools like ManyCam in front of VDO.Ninja.

steveseguin commented 2 years ago

Adding a video-only delay is quite challenging. With audio I use webaudio nodes, but there is no equal when dealing with video.

I believe can currently do it with the &chunked mode, but that mode isn't mature enough yet for production use. I do have an idea of how I might be able to modify it though to act as a local buffer source instead; I'll explore it as an option.

I believe I can also do it with a canvas draw function, however this reduces video quality and increases frame stability issues. I don't really suggest this as an option, but I can hack it out pretty easily.

I'll explore the &buffer command more, as that is one way to add video delay currently, down the the millisecond level, however last time I checked I wasn't able to buffer just the video-- it would also buffer the audio despite my instructions to do otherwise. Maybe this bug/limitation of Chrome is fixed though; I'll explore when I get a moment.

rse commented 2 years ago

Yes, I feared that this useful feature might still not exist in VDO.Ninja as there is not an easy way to implement it. I can understand the problem, Steve.