Create MediaStreamSource

[pashmak73]

@lukasf @mcosmin222 Hi, I'm really sorry for opening this, I was going to send it to your emails, but you don't provided any email in your github.

Based on this document ( AdaptiveMediaSource should supports dynamic "urn:mpeg:dash:profile:isoff-live:2011" dash profile, but video will stops after a few seconds , I keep getting this error:

DiagnosticAvailable: (12, ResourceParsingError, Manifest, URL, application/dash+xml , System.Exception: The Media format is recognized but is invalid. (Exception from HRESULT: 0xC00D3E8C))

Even I created a tiny XML parser and used it in AdaptiveMediaSourceDownloadRequestedEventArgs to replace my edited version but the error is same. So I decided to create a custom MediaStreamSource for dash live.

I tried a tiny sample which will create MediaStreamSource from a mp3 file. I tried to extend this example to support video profile as well, but I failed. here this is my code>

        private StorageFile inputMP4File;
        private StorageFile inputMP3File;
        private ulong videoByteOffset;
        private TimeSpan videoTimeOffset;
        private TimeSpan videoSampleDuration = new TimeSpan(0, 0, 0, 0, 70);

        private IRandomAccessStream videoMssStream;
        private IInputStream videoInputStream;
        private MediaStreamSource MSS = null;
        private IRandomAccessStream audioMssStream;
        private IInputStream audioInputStream;
        private ulong audioByteOffset;
        private TimeSpan audioTimeOffset;

        uint sampleRate;
        uint channelCount;
        uint bitRate;
        private const uint c_frameRateN = 30;
        private const uint c_frameRateD = 1;
        int iWidth;
        int iHeight;
        VideoStreamDescriptor _videoDesc;
        AudioStreamDescriptor _audioDesc;
        async private void Populate_Click(object sender, RoutedEventArgs e)
        {
            AudioEncodingProperties audioProps = AudioEncodingProperties.CreateMp3(sampleRate, channelCount, bitRate);
            _audioDesc = new AudioStreamDescriptor(audioProps);

            VideoEncodingProperties videoProperties = VideoEncodingProperties.CreateUncompressed(MediaEncodingSubtypes.Bgra8, (uint)iWidth, (uint)iHeight);
            //VideoEncodingProperties videoProperties = VideoEncodingProperties.CreateH264();

            _videoDesc = new VideoStreamDescriptor(videoProperties);
            _videoDesc.EncodingProperties.FrameRate.Numerator = c_frameRateN;
            _videoDesc.EncodingProperties.FrameRate.Denominator = c_frameRateD;
            _videoDesc.EncodingProperties.Bitrate = 45000;
            _videoDesc.EncodingProperties.Height = (uint)iHeight;
            _videoDesc.EncodingProperties.Width = (uint)iWidth;
            MSS = new MediaStreamSource(_videoDesc, _audioDesc);
            MSS.SampleRequested += MSS_SampleRequested;

            if (inputMP3File != null)
                audioMssStream = await inputMP3File.OpenAsync(FileAccessMode.Read);

            if (inputMP4File != null)
                videoMssStream = await inputMP4File.OpenAsync(FileAccessMode.Read);
            if (MediaPlayer == null)
            {
                MediaPlayer = new MediaPlayer();
                mediaPlayer.SetMediaPlayer(new MediaPlayer());
            }
            mediaPlayer.MediaPlayer.Source = MediaSource.CreateFromMediaStreamSource(MSS);
            mediaPlayer.MediaPlayer.Play();
        }
        async void MSS_SampleRequested(MediaStreamSource sender, MediaStreamSourceSampleRequestedEventArgs args)
        {
            MediaStreamSourceSampleRequest request = args.Request;
            if (request.StreamDescriptor is AudioStreamDescriptor)
            {
                if (audioByteOffset + audioSampleSize <= audioMssStream.Size)
                {
                    MediaStreamSourceSampleRequestDeferral deferal = request.GetDeferral();
                    audioInputStream = audioMssStream.GetInputStreamAt(audioByteOffset);
                    MediaStreamSample sample = await MediaStreamSample.CreateFromStreamAsync(audioInputStream, audioSampleSize, audioTimeOffset);
                    sample.Duration = audioSampleDuration;
                    sample.KeyFrame = true;
                    audioByteOffset += audioSampleSize;
                    audioTimeOffset = audioTimeOffset.Add(audioSampleDuration);
                    request.Sample = sample;
                    deferal.Complete();
                }
            }
           // when I uncomment this part, video won't play at all
            //else if (request.StreamDescriptor is VideoStreamDescriptor)
            //{
            //    if (videoByteOffset + audioSampleSize <= videoMssStream.Size)
            //    {
            //        MediaStreamSourceSampleRequestDeferral deferal = request.GetDeferral();
            //        videoInputStream = videoMssStream.GetInputStreamAt(videoByteOffset);
            //        MediaStreamSample sample = await MediaStreamSample.CreateFromStreamAsync(videoInputStream, audioSampleSize, videoTimeOffset);
            //        sample.Duration = videoSampleDuration;
            //        sample.KeyFrame = true;
            //        videoByteOffset += audioSampleSize;
            //        videoTimeOffset = videoTimeOffset.Add(videoSampleDuration);
            //        request.Sample = sample;
            //        deferal.Complete();
            //    }
            //}
            else
                request.Sample = null;
        }

When I uncomment the VideoStreamDescriptor part, audio/video won't play.

I put a breakpoint in FFmpegInterop to follow the building, but I got confused in the first place, that's why I decided to ask you.

Am I in a right path or not?! Can you guys help me?

Thanks in advance.

[brabebhin]

I think you need different sample sizes for audio and video In the commented code I can see

MediaStreamSample sample = await MediaStreamSample.CreateFromStreamAsync(videoInputStream, audioSampleSize, videoTimeOffset);

[lukasf]

Each video sample usually has a different size. Plus, in a DASH stream, it is packed into a MP4 container stream (and even worse, that container stream is split into lots of small chunks, each only few seconds long). I don't believe that you really want to build your own MP4 parser, do you? You could parse the XML files yourself and then use FFmpeg to play the actual streams (have one FFmpegInterop instance per stream). I don't want to demotivate you, but even that would be a whole lot of work. Implementing automatic continuous downloading of chunks, buffering, and on top of that add automatic stream switching...

[pashmak73]

Thanks for quick answers.

It seems its possible! I found a MS Sample which uses MseStreamSource for dash. MS Sample had some bugs and I fixed it for my project. Here this is the sample project for anyone who's looking for this in the future: https://docs.microsoft.com/en-us/samples/microsoft/windows-universal-samples/livedash/

I did check this example a long time ago, I never thought about it that some day it will solve my problem.

Just one thing is a pain, some codecs is not supported by the OS.