search

EnvelopSound / Earshot

Containerized RTMP->DASH transcoder for live-streaming Higher-Order Ambisonic and other multichannel content
GNU General Public License v2.0
67 stars 7 forks source link

using dash player with stream causes error messages #25

Open gschian0 opened 3 years ago

gschian0 commented 3 years ago

I've wired the dash MediaPlayer into a 3d sketch playing back on a plane... when I use the demo link and this code... I am able to see the playback on the plane...

`import { MediaPlayer } from 'dashjs';

const url = "https://dash.akamaized.net/envivio/Envivio-dash2/manifest.mpd"; const player = MediaPlayer().create(); player.initialize(document.getElementById("video"), url, true);`

when I use the current steam manifest ... "http://localhost/dash/stream1.mpd"

i get this error... Debug.js:55 [835][StreamController] Video Element Error: MEDIA_ERR_SRC_NOT_SUPPORTED (PIPELINE_ERROR_INITIALIZATION_FAILED) doLog @ Debug.js:55 fatal @ Debug.js:54 onPlaybackError @ StreamController.js:69 (anonymous) @ EventBus.js:31 trigger @ EventBus.js:31 onPlaybackError @ PlaybackController.js:71 error (async) addEventListener @ VideoModel.js:46 addAllListeners @ PlaybackController.js:146 initialize @ PlaybackController.js:30 switchStream @ StreamController.js:54 composeStreams @ StreamController.js:63 onTimeSyncCompleted @ StreamController.js:66 (anonymous) @ EventBus.js:31 trigger @ EventBus.js:31 _completeTimeSyncSequence @ TimeSyncController.js:151 _onComplete @ TimeSyncController.js:84 (anonymous) @ TimeSyncController.js:62 onload @ TimeSyncController.js:132 load (async) _httpHandler @ TimeSyncController.js:133 _attemptRecursiveSync @ TimeSyncController.js:61 attemptSync @ TimeSyncController.js:45 onManifestUpdated @ StreamController.js:69 (anonymous) @ EventBus.js:31 trigger @ EventBus.js:31 update @ ManifestUpdater.js:18 onManifestLoaded @ ManifestUpdater.js:18 (anonymous) @ EventBus.js:31 trigger @ EventBus.js:31 onXlinkReady @ ManifestLoader.js:1 (anonymous) @ EventBus.js:31 trigger @ EventBus.js:31 onXlinkAllElementsLoaded @ XlinkController.js:41 resolve @ XlinkController.js:35 onXlinkAllElementsLoaded @ XlinkController.js:40 resolve @ XlinkController.js:35 resolveManifestOnLoad @ XlinkController.js:34 success @ ManifestLoader.js:15 onload @ HTTPLoader.js:37 load (async) load @ XHRLoader.js:7 internalLoad @ HTTPLoader.js:39 load @ HTTPLoader.js:45 load @ URLLoader.js:35 load @ ManifestLoader.js:3 load @ StreamController.js:69 initializePlayback @ MediaPlayer.js:911 attachSource @ MediaPlayer.js:822 initialize @ MediaPlayer.js:85 (anonymous) @ script.js:102 (anonymous) @ bundle.2e6779f5a5db395a13f7.js:93756 (anonymous) @ bundle.2e6779f5a5db395a13f7.js:93943 Debug.js:55 [835][StreamController] [object MediaError] doLog @ Debug.js:55 fatal @ Debug.js:54 onPlaybackError @ StreamController.js:69 (anonymous) @ EventBus.js:31 trigger @ EventBus.js:31 onPlaybackError @ PlaybackController.js:71 error (async) addEventListener @ VideoModel.js:46 addAllListeners @ PlaybackController.js:146 initialize @ PlaybackController.js:30 switchStream @ StreamController.js:54 composeStreams @ StreamController.js:63 onTimeSyncCompleted @ StreamController.js:66 (anonymous) @ EventBus.js:31 trigger @ EventBus.js:31 _completeTimeSyncSequence @ TimeSyncController.js:151 _onComplete @ TimeSyncController.js:84 (anonymous) @ TimeSyncController.js:62 onload @ TimeSyncController.js:132 load (async) _httpHandler @ TimeSyncController.js:133 _attemptRecursiveSync @ TimeSyncController.js:61 attemptSync @ TimeSyncController.js:45 onManifestUpdated @ StreamController.js:69 (anonymous) @ EventBus.js:31 trigger @ EventBus.js:31 update @ ManifestUpdater.js:18 onManifestLoaded @ ManifestUpdater.js:18 (anonymous) @ EventBus.js:31 trigger @ EventBus.js:31 onXlinkReady @ ManifestLoader.js:1 (anonymous) @ EventBus.js:31 trigger @ EventBus.js:31 onXlinkAllElementsLoaded @ XlinkController.js:41 resolve @ XlinkController.js:35 onXlinkAllElementsLoaded @ XlinkController.js:40 resolve @ XlinkController.js:35 resolveManifestOnLoad @ XlinkController.js:34 success @ ManifestLoader.js:15 onload @ HTTPLoader.js:37 load (async) load @ XHRLoader.js:7 internalLoad @ HTTPLoader.js:39 load @ HTTPLoader.js:45 load @ URLLoader.js:35 load @ ManifestLoader.js:3 load @ StreamController.js:69 initializePlayback @ MediaPlayer.js:911 attachSource @ MediaPlayer.js:822 initialize @ MediaPlayer.js:85 (anonymous) @ script.js:102 (anonymous) @ bundle.2e6779f5a5db395a13f7.js:93756 (anonymous) @ bundle.2e6779f5a5db395a13f7.js:93943 Debug.js:55 [836][SourceBufferSink][video] Failed to remove source buffer from media source. doLog @ Debug.js:55 error @ Debug.js:54 reset @ SourceBufferSink.js:43 resetInitialSettings @ BufferController.js:77 reset @ BufferController.js:77 reset @ StreamProcessor.js:33 deactivate @ Stream.js:10 resetInitialSettings @ Stream.js:12 reset @ Stream.js:12 reset @ StreamController.js:69 onPlaybackError @ StreamController.js:69 (anonymous) @ EventBus.js:31 trigger @ EventBus.js:31 onPlaybackError @ PlaybackController.js:71 error (async) addEventListener @ VideoModel.js:46 addAllListeners @ PlaybackController.js:146 initialize @ PlaybackController.js:30 switchStream @ StreamController.js:54 composeStreams @ StreamController.js:63 onTimeSyncCompleted @ StreamController.js:66 (anonymous) @ EventBus.js:31 trigger @ EventBus.js:31 _completeTimeSyncSequence @ TimeSyncController.js:151 _onComplete @ TimeSyncController.js:84 (anonymous) @ TimeSyncController.js:62 onload @ TimeSyncController.js:132 load (async) _httpHandler @ TimeSyncController.js:133 _attemptRecursiveSync @ TimeSyncController.js:61 attemptSync @ TimeSyncController.js:45 onManifestUpdated @ StreamController.js:69 (anonymous) @ EventBus.js:31 trigger @ EventBus.js:31 update @ ManifestUpdater.js:18 onManifestLoaded @ ManifestUpdater.js:18 (anonymous) @ EventBus.js:31 trigger @ EventBus.js:31 onXlinkReady @ ManifestLoader.js:1 (anonymous) @ EventBus.js:31 trigger @ EventBus.js:31 onXlinkAllElementsLoaded @ XlinkController.js:41 resolve @ XlinkController.js:35 onXlinkAllElementsLoaded @ XlinkController.js:40 resolve @ XlinkController.js:35 resolveManifestOnLoad @ XlinkController.js:34 success @ ManifestLoader.js:15 onload @ HTTPLoader.js:37 load (async) load @ XHRLoader.js:7 internalLoad @ HTTPLoader.js:39 load @ HTTPLoader.js:45 load @ URLLoader.js:35 load @ ManifestLoader.js:3 load @ StreamController.js:69 initializePlayback @ MediaPlayer.js:911 attachSource @ MediaPlayer.js:822 initialize @ MediaPlayer.js:85 (anonymous) @ script.js:102 (anonymous) @ bundle.2e6779f5a5db395a13f7.js:93756 (anonymous) @ bundle.2e6779f5a5db395a13f7.js:93943 Debug.js:55 [837][SourceBufferSink][audio] Failed to remove source buffer from media source. doLog @ Debug.js:55 error @ Debug.js:54 reset @ SourceBufferSink.js:43 resetInitialSettings @ BufferController.js:77 reset @ BufferController.js:77 reset @ StreamProcessor.js:33 deactivate @ Stream.js:10 resetInitialSettings @ Stream.js:12 reset @ Stream.js:12 reset @ StreamController.js:69 onPlaybackError @ StreamController.js:69 (anonymous) @ EventBus.js:31 trigger @ EventBus.js:31 onPlaybackError @ PlaybackController.js:71 error (async) addEventListener @ VideoModel.js:46 addAllListeners @ PlaybackController.js:146 initialize @ PlaybackController.js:30 switchStream @ StreamController.js:54 composeStreams @ StreamController.js:63 onTimeSyncCompleted @ StreamController.js:66 (anonymous) @ EventBus.js:31 trigger @ EventBus.js:31 _completeTimeSyncSequence @ TimeSyncController.js:151 _onComplete @ TimeSyncController.js:84 (anonymous) @ TimeSyncController.js:62 onload @ TimeSyncController.js:132 load (async) _httpHandler @ TimeSyncController.js:133 _attemptRecursiveSync @ TimeSyncController.js:61 attemptSync @ TimeSyncController.js:45 onManifestUpdated @ StreamController.js:69 (anonymous) @ EventBus.js:31 trigger @ EventBus.js:31 update @ ManifestUpdater.js:18 onManifestLoaded @ ManifestUpdater.js:18 (anonymous) @ EventBus.js:31 trigger @ EventBus.js:31 onXlinkReady @ ManifestLoader.js:1 (anonymous) @ EventBus.js:31 trigger @ EventBus.js:31 onXlinkAllElementsLoaded @ XlinkController.js:41 resolve @ XlinkController.js:35 onXlinkAllElementsLoaded @ XlinkController.js:40 resolve @ XlinkController.js:35 resolveManifestOnLoad @ XlinkController.js:34 success @ ManifestLoader.js:15 onload @ HTTPLoader.js:37 load (async) load @ XHRLoader.js:7 internalLoad @ HTTPLoader.js:39 load @ HTTPLoader.js:45 load @ URLLoader.js:35 load @ ManifestLoader.js:3 load @ StreamController.js:69 initializePlayback @ MediaPlayer.js:911 attachSource @ MediaPlayer.js:822 initialize @ MediaPlayer.js:85 (anonymous) @ script.js:102 (anonymous) @ bundle.2e6779f5a5db395a13f7.js:93756 (anonymous) @ bundle.2e6779f5a5db395a13f7.js:93943 Debug.js:55 [839][VideoModel] Caught pending play exception - continuing (NotSupportedError: Failed to load because no supported source was found.)

are there specific settings I need to enable to handle this feed? To be thourough ... here is the html as well for the video tag... ` <video id="video" style="display: none" autoplay playsinline controls

`

feel like I'm getting pretty close to at least getting the steam into three.js

roddylindsay commented 3 years ago

hard to read these logs...but I do see an issue consistently where the stream does not properly load within the first 45 seconds or so of starting the livestream. This may be an issue with the DASH muxer with ffmpeg, needs more investigation

gschian0 commented 3 years ago

I’m going to keep playing with some settings tonight and tomorrow see if I can figure it out. Can you point me to the spot in the webtools code that implements the video stream in that window? Maybe I can adapt that into what I’m doing? I’m working on accessing the video stream first before attempting to figure out how to deal with the audio, If i can access the stream like that, i could convert it into a texture and map it to an object, I’ve already got it working with the demo link.

Gennaro Schiano 248-310-5944 @.*** VYBEZONE LLC. 360 VYBEZ www.schianosound.com

On May 13, 2021, at 7:03 PM, Roddy Lindsay @.***> wrote:

 hard to read these logs...but I do see an issue consistently where the stream does not properly load within the first 45 seconds or so of starting the livestream. This may be an issue with the DASH muxer with ffmpeg, needs more investigation

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

gschian0 commented 3 years ago

hey... some more experiments and I'm not 100% sure where to go from here... this is the code I'm using to try to get the video into my project. to get the video texture into three.js... i coppied your settings from the DashPlayer.js file ... with the other url test... it works both ways with either the settings updated or not aside from having to set the useSuggestedPresentationDelay to true... `const STABLE_BUFFER_TIME = 20; const BUFFER_TIME_AT_TOP_QUALITY = 20; const MANIFEST_LOAD_RETRY_INTERVAL = 50000; const settings = { streaming: { useSuggestedPresentationDelay: false, lowLatencyEnabled: false, stableBufferTime: STABLE_BUFFER_TIME, bufferTimeAtTopQualityLongForm: BUFFER_TIME_AT_TOP_QUALITY, retryIntervals: { MPD: MANIFEST_LOAD_RETRY_INTERVAL, }, retryAttempts: { MPD: 3, }, }, }; //this url will work no problem // const url = "https://dash.akamaized.net/envivio/Envivio-dash2/manifest.mpd"; const url = "http://localhost/dash/stream1.mpd"; const player = MediaPlayer().create(); player.initialize(document.getElementById("video"), url, true); player.updateSettings(settings);

player.isMuted = true; player.setAutoPlay(true); console.log(player);

const video = document.getElementById("video"); const videoTexture = new THREE.VideoTexture(video);`

these are the console errors... [StreamController] Video Element Error: MEDIA_ERR_SRC_NOT_SUPPORTED (PIPELINE_ERROR_INITIALIZATION_FAILED) [StreamController] [object MediaError] [SourceBufferSink][video] Failed to remove source buffer from media source. [SourceBufferSink][audio] Failed to remove source buffer from media source. here are the messages from docker... nginx-rtmp_1 | 2021/05/14 17:03:31 [info] 17#17: *30 client 172.18.0.1 closed keepalive connection nginx-rtmp_1 | 2021/05/14 17:03:31 [info] 17#17: *31 client prematurely closed connection while sending response to client, client: 172.18.0.1, server: , request: "GET /dash/chunk-stream0-00144.m4s HTTP/1.1", host: "localhost", referrer: "http://10.0.0.140:8080/" nginx-rtmp_1 | 172.18.0.1 - - [14/May/2021:17:03:31 +0000] "GET /dash/chunk-stream0-00144.m4s HTTP/1.1" 200 974672 "http://10.0.0.140:8080/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.212 Safari/537.36" nginx-rtmp_1 | 172.18.0.1 - - [14/May/2021:17:03:33 +0000] "GET /stat HTTP/1.1" 200 1604 "http://localhost/webtools/?settings={%22streaming%22:{%22useSuggestedPresentationDelay%22:false,%22lowLatencyEnabled%22:false,%22stableBufferTime%22:20,%22bufferTimeAtTopQualityLongForm%22:20,%22retryIntervals%22:{%22MPD%22:50000},%22retryAttempts%22:{%22MPD%22:3}}}" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.212 Safari/537.36" nginx-rtmp_1 | 172.18.0.1 - - [14/May/2021:17:03:38 +0000] "GET /stat HTTP/1.1" 200 1604 "http://localhost/webtools/?settings={%22streaming%22:{%22useSuggestedPresentationDelay%22:false,%22lowLatencyEnabled%22:false,%22stableBufferTime%22:20,%22bufferTimeAtTopQualityLongForm%22:20,%22retryIntervals%22:{%22MPD%22:50000},%22retryAttempts%22:{%22MPD%22:3}}}" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.212 Safari/537.36" nginx-rtmp_1 | 172.18.0.1 - - [14/May/2021:17:03:43 +0000] "GET /stat HTTP/1.1" 200 1602 "http://localhost/webtools/?settings={%22streaming%22:{%22useSuggestedPresentationDelay%22:false,%22lowLatencyEnabled%22:false,%22stableBufferTime%22:20,%22bufferTimeAtTopQualityLongForm%22:20,%22retryIntervals%22:{%22MPD%22:50000},%22retryAttempts%22:{%22MPD%22:3}}}" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.212 Safari/537.36" nginx-rtmp_1 | 172.18.0.1 - - [14/May/2021:17:03:48 +0000] "GET /stat HTTP/1.1" 200 1602 "http://localhost/webtools/?settings={%22streaming%22:{%22useSuggestedPresentationDelay%22:false,%22lowLatencyEnabled%22:false,%22stableBufferTime%22:20,%22bufferTimeAtTopQualityLongForm%22:20,%22retryIntervals%22:{%22MPD%22:50000},%22retryAttempts%22:{%22MPD%22:3}}}" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.212 Safari/537.36" nginx-rtmp_1 | 172.18.0.1 - - [14/May/2021:17:03:53 +0000] "GET /stat HTTP/1.1" 200 1604 "http://localhost/webtools/?settings={%22streaming%22:{%22useSuggestedPresentationDelay%22:false,%22lowLatencyEnabled%22:false,%22stableBufferTime%22:20,%22bufferTimeAtTopQualityLongForm%22:20,%22retryIntervals%22:{%22MPD%22:50000},%22retryAttempts%22:{%22MPD%22:3}}}" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.212 Safari/537.36" nginx-rtmp_1 | 172.18.0.1 - - [14/May/2021:17:03:58 +0000] "GET /stat HTTP/1.1" 200 1602 "http://localhost/webtools/?settings={%22streaming%22:{%22useSuggestedPresentationDelay%22:false,%22lowLatencyEnabled%22:false,%22stableBufferTime%22:20,%22bufferTimeAtTopQualityLongForm%22:20,%22retryIntervals%22:{%22MPD%22:50000},%22retryAttempts%22:{%22MPD%22:3}}}" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.212 Safari/537.36" nginx-rtmp_1 | 172.18.0.1 - - [14/May/2021:17:04:03 +0000] "GET /stat HTTP/1.1" 200 1606 "http://localhost/webtools/?settings={%22streaming%22:{%22useSuggestedPresentationDelay%22:false,%22lowLatencyEnabled%22:false,%22stableBufferTime%22:20,%22bufferTimeAtTopQualityLongForm%22:20,%22retryIntervals%22:{%22MPD%22:50000},%22retryAttempts%22:{%22MPD%22:3}}}" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.212 Safari/537.36" nginx-rtmp_1 | 172.18.0.1 - - [14/May/2021:17:04:08 +0000] "GET /stat HTTP/1.1" 200 1604 "http://localhost/webtools/?settings={%22streaming%22:{%22useSuggestedPresentationDelay%22:false,%22lowLatencyEnabled%22:false,%22stableBufferTime%22:20,%22bufferTimeAtTopQualityLongForm%22:20,%22retryIntervals%22:{%22MPD%22:50000},%22retryAttempts%22:{%22MPD%22:3}}}" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.212 Safari/537.36" thanks for following up and checking this stuff out. Once I can get everything loaded in I can start working on my player!

roddylindsay commented 3 years ago

Hi @gschian0 , unfortunately support for this will vary by client player / browser and my experience is that a lot of players will not work out of the box (even though the manifest files are valid per the MPEG-DASH spec)...e.g., they don't know what to do with 16+ audio channels.

I've attempted to show one implementation in Webtools using the dash.js reference player which has worked for me, which can be used as a reference for others. But unless there is a bug with the output files that does not match the official MPEG-DASH specification, I'm afraid I can't really help with your particular issue...

gschian0 commented 3 years ago

Thanks for following up. Do you have a simpler version of the webtools player minus the react "jargon". I'm trying to wrap my head around react better, but at this point the data flow can be pretty confusing for me. I'll try and read it slower and trace the code for now...

gschian0 commented 3 years ago

update... I am able to get the dash.player video output to display as a texture in three.js but isn't updating the feed... it's just a static texture for now using this code ...

`const STABLE_BUFFER_TIME = 20; const BUFFER_TIME_AT_TOP_QUALITY = 20; const MANIFEST_LOAD_RETRY_INTERVAL = 50000; const settings = { streaming: { useSuggestedPresentationDelay: false, lowLatencyEnabled: false, stableBufferTime: STABLE_BUFFER_TIME, bufferTimeAtTopQualityLongForm: BUFFER_TIME_AT_TOP_QUALITY, retryIntervals: { MPD: MANIFEST_LOAD_RETRY_INTERVAL, }, retryAttempts: { MPD: 3, }, }, }; const video = document.getElementById("video"); const AudioContext = window.AudioContext || window.webkitAudioContext; const audioContext = new AudioContext();

// i saw in the video.js file you decouple the audio from the video feed using this line. const mediaElemSource = audioContext.createMediaElementSource(video);

const url = "http://localhost/dash/stream1.mpd"; // const url = "http://localhost/dash/stream1.mpd"; const player = MediaPlayer().create(); player.initialize(document.getElementById("video"), url, false); player.updateSettings(settings); console.log(player);

player.isMuted = true; player.setAutoPlay(true);

const videoTexture = new THREE.VideoTexture(video);

videoTexture.minFilter = THREE.LinearFilter; videoTexture.magFilter = THREE.LinearFilter; videoTexture.format = THREE.RGBFormat; videoTexture.wrapS = videoTexture.wrapT = THREE.MirroredRepeatWrapping;`

will keep you posted on the updates when I actually get the video going instead of a static texture.