search

webtiming / timingsrc

Source code for timing related libraries managed by webtiming (multi-device timing CG)
GNU Lesser General Public License v3.0
157 stars 16 forks source link

Live Stream Sync #13

Open gazugafan opened 4 years ago

gazugafan commented 4 years ago

Related to Issue #4

I'm trying to synchronize a live HLS stream using a global epoch timestamp, but running into a wall. I think my issue must be either that... 1) I'm using video.js, and this isn't compatible with mediaSync for some reason. or 2) The epoch timestamps aren't being included in the stream.

So, I guess this is a two part question... 1) Do you have any idea if mediaSync should work with video.js? Should I switch to using hls.js instead? 2) I've included -f dash -min_seg_duration 1000000 -use_template 1 -use_timeline 1 in the advanced X264 encoder options in OBS, which I think just gets passed to ffmpeg. I'm not really sure how to confirm that this is working correctly, though. Do you know what I might look for to confirm that the epoch timestamps are indeed included? When I check the video's timestamp via the video.js object's getTimestamp() method, it's just returning the number of seconds the stream has been running--not a global epoch timestamp.

Really appreciate all your help!

Snarkdoof commented 4 years ago

I think video.js should work perfectly fine and I'm quite sure we've used it. You need to get to the point where currentTime of the video element is consistent (e.g. sent with the stream), as you say. I could always try '-format_options use_wallclock_as_timestamps=1' as well, which should provide epoc timestamps with the stream, but ffmpeg is a bit tricky. Could even be worth checking if you have an old version of ffmpeg, it's not the most stable software the world has experienced!

N

Dr. Njål Borch Senior scientist NORCE Tromsø, Norway

On Sun, 18 Oct 2020 at 00:50, Ken notifications@github.com wrote:

Related to Issue #4 https://github.com/webtiming/timingsrc/issues/4

I'm trying to synchronize a live HLS stream using a global epoch timestamp, but running into a wall. I think my issue must be either that...

  1. I'm using video.js, and this isn't compatible with mediaSync for some reason. or 2) The epoch timestamps aren't being included in the stream.

So, I guess this is a two part question...

  1. Do you have any idea if mediaSync should work with video.js? Should I switch to using hls.js instead?
  2. I've included -f dash -min_seg_duration 1000000 -use_template 1 -use_timeline 1 in the advanced X264 encoder options in OBS, which I think just gets passed to ffmpeg. I'm not really sure how to confirm that this is working correctly, though. Do you know what I might look for to confirm that the epoch timestamps are indeed included? When I check the video's timestamp via the video.js object's getTimestamp() method, it's just returning the number of seconds the stream has been running--not a global epoch timestamp.

Really appreciate all your help!

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/webtiming/timingsrc/issues/13, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABI7GQK6RH7AKCVORCVNGI3SLINT3ANCNFSM4SUVBKNQ .

gazugafan commented 4 years ago

Thanks again for your help. I've been trying to get global epoch timestamps included in the stream for the past couple days, but no luck. I think there's almost certainly something more fundamental that I don't understand. And definitely the issue isn't with this project, so feel free to close this if you'd like!

Just in case you want to help get me on track, though, here's where I'm at... I have an RTMP server setup on a VPS. The server software is NGINX with the RTMP module. I can stream to this server via OBS Studio, which encodes the video in H.264 using x264 (it can also use QuickSync or NVIDIA NVENC). The NGINX RTMP module breaks this RTMP stream up into TS chunks with an m3u8 playlist, and these files can then be served and finally played via video.js or hls.js. So far so good!

I haven't been able to find a way to get OBS Studio to encode the stream with global epoch timestamps, though. From what I understand, OBS Studio uses a custom build of ffmpeg and has limited support for adding in custom options. An OBS plugin named StreamFX adds more options, but nothing I've tried seems to have any effect on the timestamps. The video element's currentTime is always the number of seconds since the stream started. I think it's simply not possible to do this with any of the H.264 encoders included with OBS.

I've also been trying to use FFMPEG to either transcode the FLV files that OBS records, or alter the timestamps directly without transcoding, and then streaming the results to the RTMP server. No luck here either. I'm starting to feel like I've exhausted all possible options, and that there must be something more fundamental I'm not getting.

Like... should the NGINX RTMP module be responsible for adding the timestamps when it creates the TS segments? I didn't see any options for this. Maybe I've actually been successful, but the NGINX RTMP module always resets the starting timestamp back to 0 regardless of what timestamps were included?

Or... is it simply not possible to include the timestamps over an RTMP stream? Or maybe it's not possible in H.264? I'm grasping at straws now, so any advice is appreciated!

Snarkdoof commented 4 years ago

Absolute time is not important though - the important bit is that currentTime is consistent between clients. If you open the stream on the client and it starts on zero when the player starts, you need to find the local offset (what global time is zero on that client). If currentTime is something else (epoc or time since recording started), it can be synchronised immediately. :-) Calculating epoc might be a good idea, but more for convenience.

That being said, segmented video is not always a smooth experience, as the players are often slow to skip, buffer a lot and might even report stream info based on what they can play, not what is theoretically possible. My impression is that they have improved lately, but video encoding is always tricky.

Regards N

On Sun, 25 Oct 2020, 00:49 Ken, notifications@github.com wrote:

Thanks again for your help. I've been trying to get global epoch timestamps included in the stream for the past couple days, but no luck. I think there's almost certainly something more fundamental that I don't understand. And definitely the issue isn't with this project, so feel free to close this if you'd like!

Just in case you want to help get me on track, though, here's where I'm at... I have an RTMP server setup on a VPS. The server software is NGINX with the RTMP module. I can stream to this server via OBS Studio, which encodes the video in H.264 using x264 (it can also use QuickSync or NVIDIA NVENC). The NGINX RTMP module breaks this RTMP stream up into TS chunks with an m3u8 playlist, and these files can then be served and finally played via video.js or hls.js. So far so good!

I haven't been able to find a way to get OBS Studio to encode the stream with global epoch timestamps, though. From what I understand, OBS Studio uses a custom build of ffmpeg and has limited support for adding in custom options. An OBS plugin named StreamFX adds more options, but nothing I've tried seems to have any effect on the timestamps. The video element's currentTime is always the number of seconds since the stream started. I think it's simply not possible to do this with any of the H.264 encoders included with OBS.

I've also been trying to use FFMPEG to either transcode the FLV files that OBS records, or alter the timestamps directly without transcoding, and then streaming the results to the RTMP server. No luck here either. I'm starting to feel like I've exhausted all possible options, and that there must be something more fundamental I'm not getting.

Like... should the NGINX RTMP module be responsible for adding the timestamps when it creates the TS segments? I didn't see any options for this. Maybe I've actually been successful, but the NGINX RTMP module always resets the starting timestamp back to 0 regardless of what timestamps were included?

Or... is it simply not possible to include the timestamps over an RTMP stream? Or maybe it's not possible in H.264? I'm grasping at straws now, so any advice is appreciated!

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/webtiming/timingsrc/issues/13#issuecomment-716065242, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABI7GQPWHXARWQZBS7A4WELSMNKWXANCNFSM4SUVBKNQ .