Match content on AppleTV

[hendrik1120]

Describe the feature you'd like For people who have set their Apple TV's to Dolby Vision by default, content in HDR or SDR will look off. Wrong colors, tone mapping, etc. Match content as described by apple: Apple TV will use your selected display format to play content without alteration. Apple TV can also switch formats automatically to match the content's frame rate.

Usually this works automatically, except for anything played by Swiftfin. This could be a limitation of VLCKit, which won't be fixable or an easy solution to just pass the current format to Apple TV.

[LePips]

This is indeed a VLCKit issue and out of our hands.

[hendrik1120]

Thanks for checking, would have been nice. If you aren't aware already, weblate is broken again.

[perrinpagess]

This might be possible with the native video player. See #308

[hendrik1120]

I saw that too but I think I'll need to convert my whole media library for that. Unless there is a Tdarr template for it I'll just wait until VLCKit gets replaced by something actually useable.

[hendrik1120]

@perrinpagess If you got some instructions for me on how to convert a few TB of media with FFmpeg to make them HLS compliant, I will get back to this after my exams.

[perrinpagess]

@hendrik1120 I've never used tdarr before just command line ffmpeg. I can look into making a template for you if you prefer that over batch scripting and command line. In #308 I gave some guidance and ffmpeg scripts to make files HLS compliant. For Windows I've experimented setting up a script to convert all files in a folder automatically. If you don't understand anything I can help explain.

[hendrik1120]

A Tdarr Plugin would be really great. I use it only because I got a 2080super in my Windows pc, and I don't want to waste any money during a gpu shortage on a dedicated transcoding card. As Tdarr works cross-platform, I can just let my pc do the work. But if I understand it correctly, if my media already is in h.265 mp4 I only need to set some flags and only worry about the audio. Tdarr already puts the video in the first stream before decoding. The bigger problem would be the audio, since the formats are all over the place. Ranging from aac to truehd atmos. I really don't know much about encoding audio streams without loosing quality.

Edit: I guess just converting all audio into one format would be the best? So we need the following:

1. convert any video stream in H.265 mp4 using nvenc
2. convert audio without loosing quality and preferably keep atmos
3. get new subtitles for probably everything especially for the made up languages in GoT or LotR

[perrinpagess]

Yea audio is where it gets tricky...You could just encode every audio track in ALAC for a one click solution with no quality loss, but only Apple devices will play multichannel ALAC muxed in MP4 from my testing, and it probably wastes space for lossy audio sources (not an ALAC expert). Maybe a special profile for Swiftin to tell Jellyfin server to transcode all audio to ALAC when using the native player could be implemented so that no audio transcoding beforehand is needed. Muxing FLAC in MP4 is unsupported/experimental in the latest ffmpeg even though HLS supposedly supports it. Subtitles are a whole other mess that would require more Jellyfin server and Swiftfin optimization because the HLS subtitle formats aren't supported in MP4 with ffmpeg...Anyway, I'll look into making a tdarr or Jellyfin plugin for now.

[hendrik1120]

I just checked and the plugin would only need to set the correct flags. Plugins for video and audio already exist. There also is a plugin to remove all subtitles.

We then should probably focus on transcoding to something common like AC3 or dts and forget about lossless. Keeping Atmos if already there would be nice.

[hendrik1120]

I now enabled: Profile -> Playback -> Advanced -> Prefer fMP4-HLS Media Container Screenshot is from Safari It seems like I could now use the native player, at least for content with the correct audio format

[perrinpagess]

Okay. That's pretty easy then. Apple devices currently don't support TrueHD and keeping Atmos is not possible when transcoding TrueHD sources (ffmpeg discards Atmos when transcoding) so Atmos tracks need to be untouched eac3.

Another thing to consider is that sometimes there is an embedded AC3 track along with TrueHD that is still very high quality (ffmpeg can't compete with a $3000 dolby audio encoder), but sometimes the AC3 track is audio description for deaf people...hard to differentiate. Although I doubt most people can tell the difference between ffmpeg encoded ac3 and ac3 from a commercial encoder. That's probably outside the scope of a plugin so people who can tell the difference should do it themselves.

Sometimes forced subtitles are required because there's foreign language integral to the plot, but a subtitle stripping plugin would suffice for most cases.

[perrinpagess]

HEVC video has to be tagged hvc1 and in MP4 to be used with fMP4 and the native player (I think). Most video not explicitly stated as iOS compatible has to be manually tagged. You can use ffprobe to check the header. Testflight version 70 doesn't have the native player yet (next version will I presume) so I'm unable to test for sure. For now, testing native player compatibility is possible by pasting the title's stream url into safari and selecting view.

[hendrik1120]

Okay, so I just quickly build the latest Swiftfin version (upcoming 71) on Apple TV and I can confirm, match content works with the native player. My TV switched to HDR and my AVR to Dolby Atmos. Since my media isn't compliant, I could only listen to the audio and got a black screen.

[perrinpagess]

Awesome! If you have ffmpeg installed try this command to get video working for a single file that has ac3 or eac3 audio (you don't have to let the whole file process just press q after a few seconds or minutes).

ffmpeg -i input.mkv -c:v copy -tag:v hvc1 -c:a copy -map 0:v:0 -map 0:a:0 -map_chapters -1 output.mp4

[perrinpagess]

@hendrik1120 You can try it with this dolby vision video too no need to use ffmpeg

http://media.developer.dolby.com/DolbyVision_Atmos/mp4/iOS_P5_GlassBlowing2_3840x2160%4059.94fps_15200kbps.mp4

[hendrik1120]

I ofc chose a 20gig file, it transcoded in 10s bec gen4 ssd :D But I now have to copy it over again with gigabit lan :( I'll try the test file also..

[perrinpagess]

Lol yea gigabit lan is painful..

[hendrik1120]

Time to upgrade to 10gig :D But unfortunately still blackscreen :( Both with the 4k test file and the tagged one... Maybe I did something wrong, but I just right-clicked the demo vid and saved it to the server

[perrinpagess]

Hmm. Did you try playing the stream url in safari? If it works in Safari there might be something wrong with Swiftin. The Dev who worked on the native video player said he wouldn't take any issues on it :(

[hendrik1120]

Safari just downloads the file and then opens it with quick time. Where do I have to select view?

[perrinpagess]

I don't have access to my server so I can't send a screenshot, but it seems like that functionality is missing on Apple TV. On iOS and iPadOS when you enter the Jellyfin stream url it gives an option to "View" or "Download." Safari then starts downloading in MP4 segments dynamically (it stops when you press pause, skips downloading when you seek) and uses the native swift video player if you click view.

[hendrik1120]

Ok, I was using Safari on macOS before. But on my iPhone I only get a white screen after opening the url

[hendrik1120]

Never mind, it works now. Double tapping didn't copy the whole url... -,- So the native player in Swiftfin doesn't seem to be working for me at least.

[perrinpagess]

Okay. Once a new build comes out with native player feature I'll test more (don't have a mac to build). Might open up an issue if it still doesn't work and more people show interest.

[hendrik1120]

I hope the dev team considers the native player as this app should be as native as possible. VLCKit has to go anyway as it has too many issues. It will be down to mpv or native.

[hendrik1120]

We should also take this to the web app, as it just quietly fails while opening the file instead of switching to the native player automatically. Adding the tags to existing media should also be as easy as possible. I would still like a Tdarr Plugin to automatically handle new files and also the old ones. Is it also possible to apply the tags without copying the whole file in the process?

[LePips]

We will have both as an option. Most people's media isn't set up nicely for Apple's expectations of an HLS stream so forcing the native player isn't a good option

[hendrik1120]

@LePips Not forcing, but checking if the stream is compliant and launching the correct player. Keeping it under experimental is just fine for now, but it should be developed further. I mean, you just smashed the player in there, and it already has fewer issues than VLCkit IF it got the right media. Skipping, scanning, match content and everything already works. At least for now, it seems like a perfect alternative while vlc gets fixed or a new player gets made.

[LePips]

We aren't able to check whether a stream is "perfect" for the native player. I personally have no intention to further develop the native player until we get most of the app complete.

[hendrik1120]

Probably I am wrong, but Jellyfin shows "Codec tag: hvc1" in the web interface. Isn't it possible to pull the information from the api?

[LePips]

To make this short, no we can't. We probably won't ever do dynamic launching of the player, or at least forever have it as experimental.