VP9 video doesn't play in experimental native player

[wlabarron]

Describe the bug

1. In the application settings, under Experimental, enable the native player.
2. Play an H264-encoded video. See that it plays correctly.
3. Play a VP9-encoded video. See that the player displays an error -- on tvOS this is a red crossed-through circle, on iPadOS this is a crossed-through play icon.
4. Go back to application settings and switch the experimental native player off again. Retry playing the VP9 video, and see that it plays correctly.

Application version

1.0.1 (70)

Where did you install the app from?

App Store

Device information

Apple TV 4K, iPad Pro

OS version

tvOS 16.5, iPadOS 16.5.1

Jellyfin server version

10.8.10

[holow29]

Should be fixed by https://github.com/jellyfin/Swiftfin/pull/519. VP9 not supported by native player, so it should be transcoded but current device profile isn't requesting that.

[LePips]

Not exactly. For the native player, an HLS stream is requested and this doesn't consult the device profile: https://github.com/jellyfin/Swiftfin/blob/d71ecbb71638e6d105b66345a5705707798ae117/Shared/ViewModels/VideoPlayerViewModel.swift#L31

While that blob is from the refactor, the implementation is practically the same for what is currently shipped.

When Apple's player displays an error it means that the HLS stream is corrupted in some way - either a corrupted stream entirely or the stream returns a format that the player cannot play. I'm thinking to just say that I don't know if Jellyfin can properly give an HLS stream as requested from VP9.

However, I don't have any VP9 media so I can test with it. We should take a look at:

https://github.com/jellyfin/Swiftfin/blob/d71ecbb71638e6d105b66345a5705707798ae117/Shared/ViewModels/VideoPlayerViewModel.swift#L50-L52

[holow29]

Not exactly. For the native player, an HLS stream is requested and this doesn't consult the device profile:

What codecs does it use for the HLS stream if it isn't using the device profile? I guess it could just be remuxing always, but I didn't think this was the case for shipped version - don't honestly remember. (This is also why in my PR I changed it to not request HLS stream when DirectPlaying - otherwise there is no DirectPlay and it is always a remux via HLS; if there is a reason not to do it, discussion should probably be there.) It was only recently that HLS codec limitations were introduced server-side. Native player does not support VP9 so transcoding would need to occur for it to play properly.

[LePips]

What codecs does it use for the HLS stream if it isn't using the device profile?

It just uses the codecs of the video stream, those are the last lines I refer to. So, it doesn't reference the device profile. For that url construction I just referenced what the web does so I have no strong stance on what should be done.

This is also why in my PR I changed it to not request HLS stream when DirectPlaying - otherwise there is no DirectPlay and it is always a remux via HLS; if there is a reason not to do it, discussion should probably be there.

You actually just make it always use the passed playbackURL, instead of checking whether it is direct play or not switch urls. I do apologize that I haven't yet taken a look at your PR, but I will leave that comment and some other ones as well.

[holow29]

You actually just make it always use the passed playbackURL, instead of checking whether it is direct play or not switch urls. I do apologize that I haven't yet taken a look at your PR, but I will leave that comment and some other ones as well.

I think we're on the same page. It is my understanding that the playbackURL goes through checks for DirectPlay already (since that is what VLCKit is using as well IIRC. I haven't added any additional checks to switch URLs, but playbackURL does become transcodingURL if necessary in that logic.)

[LePips]

Oh yes, you are correct - it's just been a while