search

shaka-project / shaka-player

JavaScript player library / DASH & HLS client / MSE-EME player
Apache License 2.0
7.11k stars 1.33k forks source link

No audio on Tizen 3 with very large timestamps #1610

Closed avelad closed 2 days ago

avelad commented 5 years ago

Have you read the FAQ and checked for duplicate open issues?:

What version of Shaka Player are you using?: 2.5.0-beta

Can you reproduce the issue with our latest release version?: Yes

Can you reproduce the issue with the latest code from master?: yes

Are you using the demo app or your own custom app?: Both

If custom app, can you reproduce the issue using our demo app?: yes

What browser and OS are you using?: Tizen 3.0

What are the manifest and license server URIs?:

https://cdn.digital.com.bo/__cl/cg:default/__c/BOLIVIATV/__op/dash-default/__f/manifest.mpd

What did you do? Load the previous stream

What did you expect to happen? There are audio and video

What actually happened? There are video but no audio.

Note: in Chrome, Firefox, Opera and Edge the stream works.

joeyparrish commented 5 years ago

Confirmed. I get no audio on Tizen 2017.

joeyparrish commented 5 years ago

The container is audio/mp4, codec mp4a.40.2. According to the support test page, this combination is supported by the device. There's no obvious reason for this not to work.

Without a clear way to debug at the JS level on the device, this will be tricky to track down.

joeyparrish commented 5 years ago

I do get audio playback on other content.

joeyparrish commented 5 years ago

Debug logs show that audio has been selected and is being fed to the device. There's nothing obvious that the player is doing wrong in this case.

I'm not sure how this audio differs from the audio in our content, but ours plays and yours does not.

I suspect something about it is incompatible with the Tizen decoders, but that's only a guess. I don't have any data yet.

avelad commented 5 years ago

@joeyparrish , have you found something that may help find the problem?

joeyparrish commented 5 years ago

Not yet. Something is going wrong inside the black box that is the Tizen media stack. The only approach we have so far is to figure out how your content is encoded or packaged differently from ours, so that we could come up with a workaround you could apply on the encoding side.

So far, we don't know how your content is different. If you have any ideas, please let us know.

avelad commented 5 years ago

Same issue with:

https://livesim.dashif.org/livesim/testpic_2s/Manifest.mpd https://livesim.dashif.org/livesim/segtimeline_1/testpic_2s/Manifest.mpd https://livesim.dashif.org/livesim/segtimeline_1/utc_direct-head-ntp-sntp/testpic_2s/Manifest.mpd

avelad commented 5 years ago

@joeyparrish , can you share the commands (ffmepg and shaka-packager) used to generate https://storage.googleapis.com/shaka-live-assets/player-source.mpd ? We are investigating the issue with Edgeware and i think that it can help. Thanks!

joeyparrish commented 5 years ago

Here are the ffmpeg and Shaka Packager settings we use in our live stream.

First, we create an AAC audio loop and pipe it to UDP.

ffmpeg \
  -re -stream_loop -1 -i audio_loop.flac \
  -c:a aac -b:a 128k -f mpegts \
  udp://127.0.0.1:30000

Then, in another process, we encode video and put it together with the UDP audio loop without transcoding the audio. We're ingesting images from Gource via stdin for the video input.

ffmpeg \
  -probesize 32 -analyzeduration 0 -flags low_delay \
  \
  -thread_queue_size 2048 -fflags +genpts \
  -i udp://127.0.0.1:30000?overrun_nonfatal=1&fifo_size=1000000 \
  \
  -thread_queue_size 4096 -fflags +genpts+nobuffer -r 25 -f image2pipe -vcodec ppm -i - \
  \
  -c:a copy -f mpegts -vn -b:a 128k udp://127.0.0.1:40000 \
  \
  -c:v h264 -f mpegts -an -bf 0 -flags +cgop -g 100 -pix_fmt yuv420p \
  -b:v 2200k -vf pad=1280:720:20:20 udp://127.0.0.1:42000 \
  \
  -c:v h264 -f mpegts -an -bf 0 -flags +cgop -g 100 -pix_fmt yuv420p \
  -b:v 1000k -vf pad=1280:720:20:20,scale=768:432 udp://127.0.0.1:43000

The Packager is running in another process to ingest the UDP output from the second ffmpeg command and package it.

packager \
  'input=udp://127.0.0.1:40000,stream=audio,init_segment=player-source/audio-init.mp4,segment_template=player-source/audio-$Number$.mp4,bandwidth=128000,playlist_name=player-source-aac.m3u8,hls_group_id=aac' \
  'input=udp://127.0.0.1:42000,stream=video,init_segment=player-source/720-init.mp4,segment_template=player-source/720-video-$Number$.mp4,bandwidth=2200000,playlist_name=player-source-720-h264.m3u8' \
  'input=udp://127.0.0.1:43000,stream=video,init_segment=player-source/480-init.mp4,segment_template=player-source/480-video-$Number$.mp4,bandwidth=1000000,playlist_name=player-source-480-h264.m3u8' \
  --segment_duration 4 \
  --fragment_duration 4 \
  --minimum_update_period 4 \
  --suggested_presentation_delay 30 \
  --preserved_segments_outside_live_window 75 \
  --time_shift_buffer_depth $(( 15 * 60 + 30 )) \
  --mpd_output player-source.mpd \
  --hls_master_playlist_output player-source.m3u8 \
  --hls_playlist_type LIVE
avelad commented 5 years ago

Thanks @joeyparrish !

TobbeEdgeware commented 5 years ago

By using the possibility to change the availabilityStartTime (AST) of the DASH-IF live-source-simulator, it was possible to pinpoint that there is no audio playout when AST is older than 49.7 days.

The URLs look like

https://livesim.dashif.org/livesim/segtimeline_1/start_1538352000/testpic_2s/Manifest.mpd

or

https://livesim.dashif.org/livesim/start_1538352000/testpic_2s/Manifest.mpd

where 1538352000 is the number of seconds since epoch start (1970-01-01) (in this case it corresponds to 2018-10-01T00:00:00Z which is too old).

The 49.7days is very close to 2**32 milliseconds, so it is likely that there is an overflow bug in the player.

This bug means that no DASH live server can have a continuous working service for more than 49.7 days since the AST value is not allowed to change during a DASH session.

In any case, I think we can exclude that this issue is due to a bug in Shaka-player.

joeyparrish commented 5 years ago

@TobbeEdgeware, thank you for your assistance and your analysis. Can you please clarify one point for me?

If there is an overflow issue, do you believe that it is in Shaka Player? Or in the Tizen media stack?

If it's in the Tizen media stack, might it be possible to work around it in the player by adjusting timestampOffset at the SourceBuffer level?

TobbeEdgeware commented 5 years ago

@joeyparrish It must be in the Tizen media stack.

My guess is that the media time (time in DASH segments relative to AST) is converted into milliseconds in the media stack (although the timescale in the media itself is 48000 or similar).

I agree with you that it may be worth to see if one can shift the timestampOffset in the SourceBuffer with something like the session start time to get around the issue.

If that does not work, it should be possible to do a corresponding shift in the baseMediaDecodeTime in the tfdt box to bring the timestamps close to zero (essentially the same thing, but before the SourceBuffer).

You may then want to add the offset back when reporting the playback progress in order to report the correct DASH media time.

joeyparrish commented 5 years ago

Playing with baseMediaDecodeTime is outside of Shaka's scope IMO, but I will try a hacky proof-of-concept to adjust timestampOffset. If that works, we might be able to clean it up and incorporate it into Shaka Player. Thanks for the advice!

joeyparrish commented 5 years ago

@avelad, looking again at the timestamps, these are about 9 years away from reaching 2**53, which will be more than the integer accuracy of JavaScript's Number type and will cause failures related to the $Time$ substitution.

So even though I am going to try to workaround this issue on Tizen, please be aware of that broader limitation in JavaScript and consider reducing your timestamp values anyway.

avelad commented 5 years ago

@joeyparrish , the stream posted previously (https://cdn.digital.com.bo/__cl/cg:default/__c/BOLIVIATV/__op/dash-default/__f/manifest.mpd) is from Edgeware (@TobbeEdgeware ) . The previous analysis indicate that the stream only works if AST less than 49.7 days, I think that it's not possible works with these value in a production environment.

Please, tell us if it is possible to carry out the solution proposed by Tobbe.

Note: the same stream works in Opera, Chrome, Firefox, Safari, Edge and WebOS.

joeyparrish commented 5 years ago

I tried making changes in the DASH parser that would adjust timestamps artificially to work around Tizen's bug, but I was unsuccessful. Those changes in the DASH parser (effectively lying about the media timestamps) triggered a cascade of other problems in PresentationTimeline, StreamingEngine, Playhead, etc.

My suggestion at this point would be to file a bug against Tizen and see if they can issue a patch to correct their support of large audio timestamps ( > 2**32 ms ).

TobbeEdgeware commented 5 years ago

@joeyparrish Thanks for your efforts. It seems to be too much trouble to make a consistent behaviour when shifting the timestamps, so it should better be fixed in Tizen.

We have a newer segmenter where we can change the AST to an arbitrary value, which should make it possible to run the service with unpatched Tizen for at least a month before changing AST.

Regarding your comment about 53-bit Javascript integer precision, we are aware that it is an issue when using the 10,000,000 timescale of SmoothStreaming. That is the reason that we use 2000-01-01 as AST instead of 1970-01-01. With the 2000-01-01 AST, the source plays nicely in Shaka-player on other platforms.

chrisfillmore commented 5 years ago

@joeyparrish you mentioned earlier "Without a clear way to debug at the JS level on the device, this will be tricky to track down."

Have you been able to deploy a Tizen application running a Chrome debugger? It is possible. I think I previously wrote a guide on how to set this up, I can dig it up if it would be helpful.

chrisfillmore commented 5 years ago

@avelad Are your streams still available for testing? My team is looking at this issue as well and we are hoping to get Samsung to follow up. If your streams are up, they can probably use them for testing too.

avelad commented 5 years ago

@chrisfillmore , you can use: https://livesim.dashif.org/livesim/testpic_2s/Manifest.mpd https://livesim.dashif.org/livesim/segtimeline_1/testpic_2s/Manifest.mpd https://livesim.dashif.org/livesim/segtimeline_1/utc_direct-head-ntp-sntp/testpic_2s/Manifest.mpd

My stream is geoblocked

chrisfillmore commented 5 years ago

Thank you, @avelad.

We have this issue as well but, curiously, we only see the problem on 2018 TV's with our streams. Our 2017 TV's are apparently unaffected. Are you able to confirm what model year TV you are using?

My understanding of the situation is that Samsung is aware of the problem and a fix is planned for a future firmware update (which I think happen roughly quarterly).

avelad commented 5 years ago

@chrisfillmore Tizen 3 (2017) UE40MU6105 - Firmware 1250.5 Tizen 4 (2018) UE43NU7405 - Firmware 1153.3

avelad commented 5 years ago

@chrisfillmore can you share your 2017/2018 models and firmwares?

chrisfillmore commented 5 years ago

@avelad I will need to get back to you in the new year as I am off for holidays now.

stiankraggerud commented 5 years ago

Any news on this? I have the same problem. I get video, but no sound. (Tizen4(2018) UE55Q7FNA) 1153.3

chrisfillmore commented 5 years ago

@stiankraggerud this is a Samsung bug and it's my understanding that they have a fix planned for their next firmware update cycle, which I expect in about 3 months.

david-luksch-nangu commented 5 years ago

@chrisfillmore Is the Samsung issue tracked somewhere? I tried to search for it on https://bugs.tizen.org, but didn't find it there.

chrisfillmore commented 5 years ago

I believe the Tizen open source project at tizen.org tracks the Tizen OS, but not the Samsung SMART-TV variant specifically. Issues for TV platforms aren't tracked there, as far as I know.

I do not work for Samsung so take this with a grain of salt. It's my understanding that there will be a firmware update around May timeframe which will address the audio loss issue.

mafh commented 5 years ago

Hi everyone. I'm investigating Tizen (4.0) sound issue too. It doesn't produce sound on streams with availabilityStartTime = 49.72 days ago and more. The issue reproduced with dashjs, shakajs, nexplayer, etc. In other words when using html5 video element.

//OK
https://livesim.dashif.org/livesim/start_' + daysInSecondsBeforeNow(1) + '/testpic_2s/Manifest.mpd
//FAILS: NO SOUND ISSUE
https://livesim.dashif.org/livesim/start_' + daysInSecondsBeforeNow(50) + '/testpic_2s/Manifest.mpd
//OK
https://livesim.dashif.org/livesim/segtimeline_1/start_' + daysInSecondsBeforeNow(1) + '/testpic_2s/Manifest.mpd
//FAILS: NO SOUND ISSUE
https://livesim.dashif.org/livesim/segtimeline_1/start_' + daysInSecondsBeforeNow(50) + '/testpic_2s/Manifest.mpd

Nevertheless the device produce sound when utilizing AVPlay API for simple MPD without SegmentTimelines:

//OK
https://livesim.dashif.org/livesim/start_' + daysInSecondsBeforeNow(1) + '/testpic_2s/Manifest.mpd
//OK
https://livesim.dashif.org/livesim/start_' + daysInSecondsBeforeNow(50) + '/testpic_2s/Manifest.mpd

Unfortunately, the device freezes for 80 seconds when SegmentTimeline tag presents in MPD file.

//FAILS, FREEZE 80S, {"code":17,"message":"PLAYER_ERROR_INVALID_OPERATION","name":"TypeMismatchError"}
https://livesim.dashif.org/livesim/segtimeline_1/start_' + daysInSecondsBeforeNow(1) + '/testpic_2s/Manifest.mpd
//FAILS, FREEZE 80S, {"code":17,"message":"PLAYER_ERROR_INVALID_OPERATION","name":"TypeMismatchError"}
https://livesim.dashif.org/livesim/segtimeline_1/start_' + daysInSecondsBeforeNow(50) + '/testpic_2s/Manifest.mpd

In my opinion this indicates on some overflow bug, but i still can't figure out that exactly happens.

I also can't see any flaws in a diff of two parsed mpds except segment duration value. Do you think it may be cause the issue?

mpds-diff

joeyparrish commented 5 years ago

Yes, 49.72 days is a very suspicious number: 2^32 milliseconds. This is definitely a bug in the Tizen platform.

There is a possibility to work around it, but it will not be quick or easy as far as I can tell. We would have to artificially offset long-lived streams, which would have ripple effects into many parts of the system which rely on accurate timestamps. It will be easier to try after the manifest parsers and internal structures are refactored in v2.6.

mafh commented 5 years ago

Tizen TV 4.0 has updated its firmware this morning, from 1202 to 1251. And there is sound available on live streams, any days long. Nothing left to say. http://livesim.dashif.org/livesim/start_4320000/testpic_2s/Manifest.mpd //ok 100 days long

segmentTimeline is still a fiasco for avplay api. but shaka manages to play the same stream https://livesim.dashif.org/livesim/segtimeline_1/start_4320000/testpic_2s/Manifest.mpd

avelad commented 5 years ago

I can confirm that in Tizen 4.0 is working now!

joeyparrish commented 5 years ago

Woohoo! Wonderful news!

MLangendijk commented 2 years ago

@joeyparrish I'm still experiencing this issue on Samsung 2017 televisions with latest firmware. My client can't change the stream AvailabilityStarttime sadly. You mentioned a potential artificial offset, is that something you can give some pointers to that I can work on myself as a local experiment?

avelad commented 1 week ago

No one from Shaka's team is going to work on this. If anyone is interested please create a PR, otherwise the issue will be closed in 7 days.

shaka-bot commented 2 days ago

Closing due to inactivity. If this is still an issue for you or if you have further questions, the OP can ask shaka-bot to reopen it by including @shaka-bot reopen in a comment.