In the following example, a new MediaPlayer is initialized and video content is loaded. Next, a handler for VideoFrameAvailable is registered. Frame server mode is enabled by setting the MediaPlayer object's IsVideoFrameServerEnabled property to true. Finally, media playback is started with a call to Play.
ref #953
プログレッシブダウンロードmp4前提の動画キャッシュからHLSの動画キャッシュへと移行が必要です。
[2023/07/12 追記]
MediaPlayer の Frame Server Mode を利用すれば対応できそうです。
https://learn.microsoft.com/en-us/windows/uwp/audio-video-camera/play-audio-and-video-with-mediaplayer#use-mediaplayer-in-frame-server-mode