10-bit H.264 and 4:4:4 H.264 video freezes process on closing media

Describe the bug This bug and problem is divided into 2 parts:

The main part is, if we try to open a 10-bit video file, it will crash the Unity editor (when you exit play mode) and the standalone build (if you try to close the app). And this is when I have tried both playing the video with _mediaplayer.Play() and without Playing the video and just trying to OpenMedia and access its videotrack data in MetaDataReady event. Tested with both MediaFoundation (yeah I know it doesnt work) and DirectShow API with LAVFilters installed, even tested by ticking the 10-bit option in Windows platform.
Really need a way to detect 10-bit videos. There is literally no way if using MediaFoundation to do that, but using DirectShow there is a way by reading the videoTrackName.
If using MediaFoundation (Win10), there is absolutely no way to detect or tell if a video is 10bit and there is no way to handle the crash which happens as soon as you _mediaplayer.stop or exit play mode. Furthermore, in MetaDataReady event, doing _mediaPlayer.VideoTracks.GetActiveVideoTrack().DisplayName only gets you something like _track(en) or something generic. This is frustrating. If using DirectShow (LAVFilters installed), 10-bit video still does not play even though am on Windows 10 and have LAVFilters installed and DirectShow API selected (no other option modified). However, trying to get video's name in DirectShow actually provides more detail including the part "yuv**" which I think can be used to detect 10bit video and close video player right there and then and maybe show a error message. Below is the video track name returned: V: English [eng] (h264 high 10 L5.0, yuv420p10le, 1280x720) [default] (en-US)

^^ This can still work if not for this weird crash reported in #1, as my plan was to use DirectShow API and not play the media immediately, instead wait for MetaDataReady event, check for videotrack name if it has "yuv420p10le" in it and if yes, exit the videoplayer and show the user a notification/error.

However, not only the video still cannot be played, even then the crash problem is there even with DirectShow.

Your Setup (please complete the following information):

Unity version: 2020.2
AVPro Video version: AVPro Windows 2.1.0
Operating system version: Windows 10 x64
Device model: Desktop PC (Unity Editor/Standalone Build)

To Reproduce

Open the provided 10-bit video with DirectShow + LAV (just open, no need to even play as its secondary priority for now, only need to access video track details).
Either try to play the video or simply stop the editor (exit play mode) or close your build.

Logs I am not attaching the logs as there is nothing specific in them, no crash reports, no errors, no exceptions, as most of the stuff I am debu.log printing is the Events, the last log print everytime is just the EventType.Closing
Somehow I have a feeling either _mediaplayer.Stop() or the CloseMedia method is making the crash. Unfortunately no crash logs or anything usefull in either Editor or Standalone Player logs, last entry is always MediaPlayerEvent.EventType.Closing
The video is:
V: English [eng] (h264 high 10 L5.0, yuv420p10le, 1280x720) [default] (en-US)

Please DO NOT LINK / ATTACH YOUR PROJECT FILES HERE

Instead email the link to us unitysupport@renderheads.com
^^Sent you a link for the file on email with the Github issue ID in subject

Code

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.UI;
using System.Threading.Tasks;
using System.Runtime.InteropServices;
using System.IO;
using RenderHeads.Media.AVProVideo;
using UnityEngine.Events;
using System;

public class AVProManager : MonoBehaviour
{

    public static AVProManager _instance;

    internal MediaPlayer _mediaPlayer;
    internal MediaPath _video;

    const int seekTimeDelta = 10000;
    Texture2D tex = null;
    bool playing;
    bool currentMediaWasParsed = false;
    bool reset = true;                 //For changing media

    internal RawImage targetTexture;

    bool isOpen = false;

    private void Awake()
    {
        if (_instance != null)
        {
            Destroy(gameObject);
        }
        else
        {
            _instance = this;
        }
    }
    // Start is called before the first frame update
    void Start()
    {

    }

    internal void SeekForward()
    {
        Debug.Log("[AV] Seeking forward ! ");
        _mediaPlayer.Control.SeekFast((VideoPlayerManager._instance.curDur/1000) + (seekTimeDelta/1000));
    }

    internal void SeekBackward()
    {
        Debug.Log("[AV] Seeking backward !");
        _mediaPlayer.Control.SeekFast((VideoPlayerManager._instance.curDur / 1000) - (seekTimeDelta / 1000));
    }

    /*
    void OnDisable()
    {
        if (_video != null)
        {
            Debug.Log("Disposing video");
            _video = null;
        }

        _mediaPlayer?.Stop();
        _mediaPlayer = null;

        //_video.ParsedChanged -= OnParsed;
        Debug.Log("AVPro OnDisable");
    }
    */

    //Order of events: StartBuffering->MetaDataReady->FinishedBuffering->Started->FirstFrame->Stalled/Unstalled (lose focus)->Text Cue Changed->Closing->FinishedPlaying (if video finishes)
    //10Bit error order: StartBuffer->MetaDataReady->FinishBuffer->Started
    public void OnMediaPlayerEvent(MediaPlayer mp, MediaPlayerEvent.EventType et, ErrorCode errorCode)
    {
        switch (et)
        {
            case MediaPlayerEvent.EventType.Started:
                Debug.Log("Creating texture with height " + _mediaPlayer.Info.GetVideoHeight() + " and width " + _mediaPlayer.Info.GetVideoWidth());
                //Change Scale of the texture to match video aspect ratio
                VideoPlayerManager._instance.changeScreenSize(_mediaPlayer.Info.GetVideoHeight(), _mediaPlayer.Info.GetVideoWidth());              
                break;
            case MediaPlayerEvent.EventType.FinishedPlaying:
                Debug.Log("Media stopped");
                onStopped();
                break;
            case MediaPlayerEvent.EventType.Error:
                Debug.Log(errorCode.ToString() + " Error in AVPro " + (ErrorCode)errorCode);
                onPlayerError();
                VideoPlayerManager._instance._playing = false;
                this.isOpen = false;
                VideoPlayerManager._instance.stopVideo();
                break;

            case MediaPlayerEvent.EventType.Closing:
                Debug.Log("Media File closed");
                if(!VideoPlayerManager._instance._playing && !this.isOpen)
                {
                    Debug.Log("Media File loading problem");
                }
                VideoPlayerManager._instance._playing = false;
                this.isOpen = false;
                this.playing = false;
                this.reset = true;
                this.onStopped();              
                break;

            case MediaPlayerEvent.EventType.FinishedSeeking:
                break;
            case MediaPlayerEvent.EventType.MetaDataReady:
                Debug.Log("Media metadata ready");
                Debug.Log(_mediaPlayer.VideoTracks.GetActiveVideoTrack().DisplayName);
                if (_mediaPlayer.VideoTracks.GetActiveVideoTrack().DisplayName.Contains("yuv420p10le"))
                {
                    Debug.Log("Detected 10-bit video");
                    //_mediaPlayer = null;
                    //return false;
                }
                break;
            case MediaPlayerEvent.EventType.ReadyToPlay:
                Debug.Log("Media ready to play");
                break;
            case MediaPlayerEvent.EventType.Stalled:
                break;
            case MediaPlayerEvent.EventType.Unstalled:
                break;
            case MediaPlayerEvent.EventType.TextCueChanged:
                break;
            case MediaPlayerEvent.EventType.FirstFrameReady:
                Debug.Log("First frame is ready, begin playback");
                _mediaPlayer.Play();
                VideoPlayerManager._instance._playing = true;
                this.enableSubtitles();
                this.isOpen = true;
                break;
            default:
                Debug.Log((MediaPlayerEvent.EventType)et);
                break;
        }

    }

    internal void enableSubtitles()
    {
        //Enable subtitles
        if (_mediaPlayer.TextTracks.GetTextTracks().Count < 1)
        {
            Debug.Log("No subtitles detected");
        }
        else
        {
            Debug.Log("Enabling subs: " + _mediaPlayer.TextTracks.GetTextTracks()[0].DisplayName + " " + _mediaPlayer.TextTracks.GetTextTracks()[0].Language + " " + _mediaPlayer.TextTracks.GetTextTracks()[0].Name);
            _mediaPlayer.TextTracks.SetActiveTextTrack(_mediaPlayer.TextTracks.GetTextTracks()[0]);
            onPlaying();
        }
    }

    public bool PlayPause(string path = "", bool network = false)
    {

        //Debug.Log("[VLC] Toggling Play Pause !");
        if (_mediaPlayer == null)
        {
            _mediaPlayer = this.gameObject.GetComponent<RenderHeads.Media.AVProVideo.MediaPlayer>();
        }

        //if (_mediaPlayer.IsPlaying)
        if (VideoPlayerManager._instance._playing)
        {
            _mediaPlayer.Pause();
            this.onPaused();
            Debug.Log("Pause");
            VideoPlayerManager._instance._playing = false;
        }
        else
        {
            playing = true;
            Debug.Log("Play");
            if (reset)
            {
                Debug.Log("Set Media");
                if (path != "" && network)
                {
                    // playing remote media
                    path = path.Replace("[", "%5B").Replace("]", "%5D");
                    _video = new MediaPath(path, MediaPathType.AbsolutePathOrURL);

                    //_video.ParsedChanged += OnParsed;
                    //parseVideoAsync(true);
                    _mediaPlayer.OpenMedia(_video, true);
                    //Debug.Log(_mediaPlayer.Media.TrackList(TrackType.Text));
                }
                else if (path != "" && !network)
                {
                    //Debug.Log("Using local path: " + path);
                    // playing local media
                    _video = new MediaPath(path, MediaPathType.AbsolutePathOrURL);
                    Debug.Log("_video path: " + _video.Path);

                    //_video.ParsedChanged += OnParsed;
                    //_video.Parse(MediaParseOptions.ParseLocal);
                    //parseVideoAsync(false);
                    this.isOpen = _mediaPlayer.OpenMedia(_video, false);
                    Debug.Log("Bool isOpen: " + isOpen);
                    if (!isOpen)
                    {
                        Debug.Log("Error opening media");
                        ExtraScreenManager._instance.showNotification("Failed to open video file. Using "+ (Windows.VideoApi)_mediaPlayer.PlatformOptionsWindows.videoApi +" Either not running Win10 or LAVFilter not found? Report bug please.", "ERROR:");
                        return false;
                    }
                    //Debug.Log(_mediaPlayer.Media.TrackList(TrackType.Text));
                }
                //_video?.Dispose();
                //_video = null;
            }

            Debug.Log("Check If Media is ready to play");

            //First media playback is done from FirstFrameIsReadyEvent

            if (isOpen && !VideoPlayerManager._instance._playing)
            {
                Debug.Log("First playback");
                _mediaPlayer.Play();
                VideoPlayerManager._instance._playing = true;
            }

            return true;
        }

        return true;
    }

    private void onPlayerError()
    {
        UnityEngine.Debug.Log("AVPro Player error: ");
        try
        {
            if (Directory.Exists(Config._instance.getDownloadRootPath() + "/" + AnimeDetailManager._instance.parseFileName(AnimeDetailManager._instance.res.foldername)))
            {
                System.Diagnostics.ProcessStartInfo startInfo = new System.Diagnostics.ProcessStartInfo();
                startInfo.Arguments = Config._instance.getDownloadRootPath() + "/" + AnimeDetailManager._instance.parseFileName(AnimeDetailManager._instance.res.foldername);
                startInfo.FileName = "explorer.exe";
                System.Diagnostics.Process.Start(startInfo);
            }
        }
        catch (Exception ex)
        {
            UnityEngine.Debug.Log("Failure to open Anime Folder: " + ex.Message);
        }
    }

    private void onTimeChanged()
    {
        VideoPlayerManager._instance.curDur = (float)(_mediaPlayer.Control.GetCurrentTime() * 1000);
    }

    private void onStopped()
    {
        CancelInvoke("onTimeChanged");
        VideoPlayerManager._instance._playing = false;
    }

    private void onPlaying()
    {
        InvokeRepeating("onTimeChanged", 0.25f, 0.25f);
        VideoPlayerManager._instance._playing = true;
        this.reset = false;
        VideoPlayerManager._instance.totDur = (float)(_mediaPlayer.Info.GetDuration() * 1000);

    }
    private void onPaused()
    {
        VideoPlayerManager._instance._playing = false;
        Debug.Log("MediaPlayer is paused ? " + _mediaPlayer.Control.IsPaused());
    }

    public void Stop()
    {
        Debug.Log("[AVPro] Stopping Player !");

        if (_video != null)
        {
            Debug.Log("Disposing video");
            //_video?.Dispose();
            //_video = null;
        }

        playing = false;
        _mediaPlayer?.Stop();

        reset = true;
        this.onStopped();

    }

}

Also, I can confirm using AVPro Video's demo scene with a media reference and the 10-bit video has same behavior, played the demo scene, no video playback or frame.
And the moment you try to exit play mode or stop editor, boom hangs up.

Hi @shubhank008

We're currently investigating this. Are you using the Core or Ultra or Trial version of AVPro?

Hi @shubhank008.

The crash you're getting is due to the videos being H.264 + 10-bit . I know we've already spoken about this, but this table outlines which API's and codecs will work with 10-bit and which won't. We will see if we can find a solution from the crash although it might not be possible. I managed to reproduce it with your H.264 + 10-bit video, but only for Media Foundation and WinRT. Direct Show + LAV Filters plays the video fine. To confirm, you are experiencing the crash on Direct Show, using LAV set as your preferred filter in platform specific settings?

Hi @shubhank008.

The crash you're getting is due to the videos being H.264 + 10-bit . I know we've already spoken about this, but this table outlines which API's and codecs will work with 10-bit and which won't. We will see if we can find a solution from the crash although it might not be possible. I managed to reproduce it with your H.264 + 10-bit video, but only for Media Foundation and WinRT. Direct Show + LAV Filters plays the video fine. To confirm, you are experiencing the crash on Direct Show, using LAV set as your preferred filter in platform specific settings?

The support or codecs part is understandable, its just that it crashes instead of any exception handling or errors is just something that shouldn't be like that, that just means if a user ever tries to play 10bit by mistake the app will crash and he or probably even we would never know why.
So a graceful exit or exception or error would be great so that it can be handled.

And yes, I tried with Direct Show as well, in preferred filters table I believe I had the table/priority set to LAV Video, LAV Audio and then Microsoft one. <<-- But this part I might have to test again as then I had installed LAV filters using their quiet installer which I believe was not working as it should then, I later installed them again using their GUI installer but don't remember if I tested it or not.
But again, when there is no inbuilt way to detect or know if the video is 10bit, that just means using direct show as the only option to avoid crashes or detect 10-bit video using the video track name but then embedded subtitles won't work.

For now, I have created my own solution using MediaInfo .NET wrapper via which I am able to check if a video is 10bit and handle it without trying to forward/playing it with AVpro.

We encountered the exact same problem testing the 10bit support of the Ultra Edition. If a filter is chosen when using DirectShow that actually cannot decode the video nothing happens. No texture is created and no event is sent. When closing the MediaPlayer/destroying the GameObject the application/Unity Editor crashes. We acutally had different behavior on different machines. One would play all the files and one wouldn't. Checking with GraphStudioNext showed that on one machine a Microsoft decoder was chosen over the LAV one. So I added this line before opening the video MediaPlayer.PlatformOptionsWindows.preferredFilters = new List<string> {"LAV Video Decoder"}; what fixed the problem. Opening a NotchLC video using DirectShow (knowing it wouldn't work) showed similar behavior. No texture and no error event but at least it didn't crash on close. But it still leaves the application in an undefined state. The log showed this message Using playback path: DirectShow (0x0@0,00) So better error handling when using DirectShow would be very desirable. I would consider it a serious bug knowing that the application could crash when a video cannot be decoded. Using Media Foundation seems to have better error handling. In most cases an error event is sent. We still have crashes on one machine with an h264 10bit video with the same behavior as with DirectShow with no texture and no event and the crash on closing it. I also wrote a post on the forum that describes our setup: Forum Post I could provide our test videos via cloud storage link (via email) if wanted.

We encountered the exact same problem testing the 10bit support of the Ultra Edition. If a filter is chosen when using DirectShow that actually cannot decode the video nothing happens. No texture is created and no event is sent. When closing the MediaPlayer/destroying the GameObject the application/Unity Editor crashes. We acutally had different behavior on different machines. One would play all the files and one wouldn't. Checking with GraphStudioNext showed that on one machine a Microsoft decoder was chosen over the LAV one. So I added this line before opening the video MediaPlayer.PlatformOptionsWindows.preferredFilters = new List<string> {"LAV Video Decoder"}; what fixed the problem. Opening a NotchLC video using DirectShow (knowing it wouldn't work) showed similar behavior. No texture and no error event but at least it didn't crash on close. But it still leaves the application in an undefined state. The log showed this message Using playback path: DirectShow (0x0@0,00) So better error handling when using DirectShow would be very desirable. I would consider it a serious bug knowing that the application could crash when a video cannot be decoded. Using Media Foundation seems to have better error handling. In most cases an error event is sent. We still have crashes on one machine with an h264 10bit video with the same behavior as with DirectShow with no texture and no event and the crash on closing it. I also wrote a post on the forum that describes our setup: Forum Post I could provide our test videos via cloud storage link (via email) if wanted.

If your target device is Windows only for this problem (I think 10bit are handled better on mobile devices), I advise you the same as me till RenderHeads fixes it (by fix I mean better crash/error handling).
Use MediaInfo's .net library and using it you can detect 10bit and lot more.

I am even planning to create a fallback system using MediaInfo and CCExtractor where if I detect 10bit videos (currently I donot allow their playback) using MediaInfo, I will switch my player API from MediaFoundation to DirectShow.
DirectShow plays audio and video for 10bit as long as LAV installed (I have even created a check to see if LAV installed else show its installation when trying to play 10bit, shipping LAV executable with my program).

Now the problem remained that DirectShow API cannot play/show embedded subtitles, so for that I am again using MediaInfo API and CCExtractor to extract those embedded subtitles to srt files which can then be side-loaded (external subtitles) into AVpro and subtitles works with 10bit DirectShow.

Voila !!

Hi @shubhank008.

The crash you're getting is due to the videos being H.264 + 10-bit . I know we've already spoken about this, but this table outlines which API's and codecs will work with 10-bit and which won't. We will see if we can find a solution from the crash although it might not be possible. I managed to reproduce it with your H.264 + 10-bit video, but only for Media Foundation and WinRT. Direct Show + LAV Filters plays the video fine. To confirm, you are experiencing the crash on Direct Show, using LAV set as your preferred filter in platform specific settings?

@kahnivore I am facing same issue with a 8-bit mkv file now, some file plays, some don't (when playing from URL).
I have confirmed same file playing in VLC and other media player and even MediaInfo is able to fetch its data, so the server is okay and supports streaming playback, yet the video loads all the data but never plays/streams.

Again same problem, once play is pressed, nothing happens, video doesn't play or stream and when you try to stop the player or Unity Editor or standalone app, it just hangs/freezes up and does nothing unless you end task it from task manager.
Using MediaFoundation, want to use it cause of inbuilt subtitle support and as we are not targetting 10bit videos.
Running on Windows 10.

Frustrating.

URL:
https://store.animewat.ch//Gensou%20Mangekyou%20The%20Memories%20of%20Phantasm/04%20-%20The%20Scarlet%20Mist%20Incident%20Final%20Part%5B%5D%5BRapidBot%5D.mkv

MediaInfo:

General
Unique ID                                : 19234161976725216571534871560529121083 (0xE785D66D99BB7E0E8454B579DD80F3B)
Complete name                            : https://store.animewat.ch//Gensou Mangekyou The Memories of Phantasm/04 - The Scarlet Mist Incident Final Part[][RapidBot].mkv
Format                                   : Matroska
Format version                           : Version 4
File size                                : 143 MiB
Duration                                 : 18 min 47 s
Overall bit rate                         : 1 066 kb/s
Movie name                               : 04 - The Scarlet Mist Incident Chapter (Final Part)
Writing application                      : Lavf57.76.100
Writing library                          : Lavf57.76.100
ErrorDetectionType                       : Per level 1

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High 4:4:4 Predictive@L4
Format settings                          : CABAC / 6 Ref Frames
Format settings, CABAC                   : Yes
Format settings, Reference frames        : 6 frames
Codec ID                                 : V_MPEG4/ISO/AVC
Duration                                 : 18 min 47 s
Bit rate                                 : 3 411 kb/s
Width                                    : 1 280 pixels
Height                                   : 720 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 23.976 FPS
Chroma subsampling                       : 4:4:4
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 0.154
Stream size                              : 459 MiB
Title                                    : BDRip by Vladdrako
Writing library                          : x264 core 148 r2748 97eaef2
Encoding settings                        : cabac=1 / ref=6 / deblock=1:1:1 / analyse=0x3:0x113 / me=hex / subme=7 / psy=1 / psy_rd=0.40:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=4 / threads=6 / lookahead_threads=1 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=5 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=23 / scenecut=40 / intra_refresh=0 / rc_lookahead=40 / rc=crf / mbtree=1 / crf=23.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / vbv_maxrate=2000 / vbv_bufsize=2000 / crf_max=0.0 / nal_hrd=none / filler=0 / ip_ratio=1.40 / aq=1:0.60
Language                                 : Japanese
Default                                  : Yes
Forced                                   : No

Audio #1
ID                                       : 2
Format                                   : AAC LC
Format/Info                              : Advanced Audio Codec Low Complexity
Codec ID                                 : A_AAC-2
Duration                                 : 18 min 47 s
Bit rate                                 : 797 kb/s
Channel(s)                               : 2 channels
Channel layout                           : L R
Sampling rate                            : 48.0 kHz
Frame rate                               : 46.875 FPS (1024 SPF)
Compression mode                         : Lossy
Delay relative to video                  : -42 ms
Stream size                              : 107 MiB (75%)
Title                                    : LPCM->FLAC (w/o Voice)
Writing library                          : Lavc57.102.100 aac
Language                                 : Japanese
Default                                  : No
Forced                                   : No

Audio #2
ID                                       : 3
Format                                   : AAC LC
Format/Info                              : Advanced Audio Codec Low Complexity
Codec ID                                 : A_AAC-2
Duration                                 : 18 min 41 s
Bit rate                                 : 169 kb/s
Channel(s)                               : 2 channels
Channel layout                           : L R
Sampling rate                            : 44.1 kHz
Frame rate                               : 43.066 FPS (1024 SPF)
Compression mode                         : Lossy
Delay relative to video                  : 1 s 941 ms
Stream size                              : 22.6 MiB (16%)
Title                                    : Jap Dub Vivian
Writing library                          : Lavc57.102.100 aac
Language                                 : Japanese
Default                                  : Yes
Forced                                   : No

Menu
00:00:00.000                             : :OP
00:03:20.894                             : :Part A
00:08:20.234                             : :Part B
00:16:10.329                             : :ED
00:17:49.053                             : :Part C

I believe the problem is that this video is using 4:4:4 chroma sub-sampling instead of the more common 4:2:0

We don't support every single type of media - we rely on the operating system support. On Windows the Microsoft H.264 decoder only supports 4:2:0 as you can see here: https://docs.microsoft.com/en-us/windows/win32/medfound/h-264-video-decoder#format-constraints

We will update the documentation to note this.

You'll notice if they try to open that MKV video with Windows 10 "Films & TV" app, it will have the same behaviour. VLC is designed to support as many formats as possible, even if that falls back to a software decoder.

I will have a look to see if we can prevent the freezing from happening, but upon initial investigation it seems to be an issue deep in the microsoft codec.

Thanks,

Btw, on DirectShow that 4:4:4 MKV video works when using LAV Filters and adding "LAV Video Decoder" to the list of perferred DirectShow filters.

In summary (as notes to ourselves mostly), the issues are:

1) If you load a 10-bit or 4:4:4 H.264 video using Media Foundation or WinRt, the video will not display and closing the media will freeze the process. This seems to be a bug deep in Media Foundation / h.264 decoder, which doesn't support these formats. It may be possible at least to prevent a crash by creating a new custom ByteStreamHandler to pre-process .mp4 and .mkv files and reject them if they are 10-bit or 4:4:4, however I think if a custom handler is used then the other ones are no longer used as fallbacks, so I'm not sure if this is approach would really work. Another possible fix would be to pre-process the mp4/mkv before using standalone code before deciding to open the video with MediaPlayer. This may tie well into the built-in MediaInfo-like script we would like to add in the future. Also note that for 10-bit videos, the HEVC/H.265 decoder does work, it's just a limitation of Microsoft's H.264 decoder.

2) With DirectShow it's possible to play these, but only using the LAV Filters and with the LAV Video Decoder explicitly specified. The 4:4:4 H.264 on DirectShow (without LAV Video Decoder) can also result in a process lockup though when closing the media. This seems to be exactly the same behaviour as in Winrt and Media Foundation, so it does seem to be a deep problem. During the shutdown of the graph it just never returns...

I believe the problem is that this video is using 4:4:4 chroma sub-sampling instead of the more common 4:2:0

We don't support every single type of media - we rely on the operating system support. On Windows the Microsoft H.264 decoder only supports 4:2:0 as you can see here: https://docs.microsoft.com/en-us/windows/win32/medfound/h-264-video-decoder#format-constraints

We will update the documentation to note this.

You'll notice if they try to open that MKV video with Windows 10 "Films & TV" app, it will have the same behaviour. VLC is designed to support as many formats as possible, even if that falls back to a software decoder.

I will have a look to see if we can prevent the freezing from happening, but upon initial investigation it seems to be an issue deep in the microsoft codec.

Thanks,

Ah, damn. I think there needs to be more extensive list of format and codecs supported in docs, or atleast a link to indirect list like the microsoft one you linked above.

But I think the main problem and point of frustration for all of this is the crash/freeze whenever something is not supported instead of doing a graceful error or exception.
Because otherwise one can never know what's failing or if its non-supported codec, etc. and keeps crashing again and again.

So far because of non-error/crash handling I am having to use 3rd party library MediaInfo to get info about every file before trying to play it and write my own check functions to make sure user don't try to play a non-supported format (I had one function for 10-bit and now one for the chroma sampling).
Which I believe wouldn't be necessary if error/exception handling for non-supported or failed media is added.

For DirectShow, does the preferred filters need only LAV Video in the list ? Mine is LAV Video, LAV Audio and Microsoft (in order) and even then its giving same behavior and crash. The LAV icon shows in the system tray when it tries to play, but then nothing happens and it just freezes up.

I think if its a microsoft limitation or needs deep diving, then using MediaInfo is probably a better idea. This has even encouraged me to create a web-based MediaInfo API so perhaps that will be useful and can be used on PC and mobile devices both.

Solution with MediaInfo works, but it loads too much of a video to give details. For 50GiB video it reads 120MiB of it and over network it takes several seconds.

RenderHeads / UnityPlugin-AVProVideo

10-bit H.264 and 4:4:4 H.264 video freezes process on closing media #759

Please DO NOT LINK / ATTACH YOUR PROJECT FILES HERE