Black screen on iOS device

[sw5163]

Only change dvhe->dvh1 in hex will cause problem on iPad Pro 2017 and possibly all other iOS devices. https://forum.makemkv.com/forum/viewtopic.php?style=4&f=12&t=18602&start=7305#p126502

[rixtox]

Please provide more context:

1. Sample file used for reproducing the issue.
2. What player application was used on your iOS device.

[sw5163]

Thanks for the quick reply!

1. Sample file used for reproducing the issue.

All dvhe mp4 file changed this way have this problem.

2. What player application was used on your iOS device.

Apple's native player (quicktime)

I am aware of Mac can play them, but that's not the case for my iPad. From yusesope: Those header indicate to the decoder how the information for the initialization of the latter is stored within the file. Changing the header with a hex editor isn't always enough! QuickTime on Mac is perhaps smarter enough and can retrieve the information needed to initialize the decoder. The iPad player is not capable of it.

Conversion from dvhe->dvh1 by mp4box or mp4muxer will be fine like I posted on AVSForum. If you have an iOS device, you can try it on Safari/File App/Photos. All app using native player will have same problem. Trying to fix this will take a huge amount of time, I wouldn't recommend fixing it.

[rixtox]

Some suggestion on Apple platform workflows. Video playback has two modes with slightly different software stacks on Apple platforms: file playback and streaming playback.

The dvhe->dvh1 is a QuickTime file playback conformance hack, where only dvh1 is recognized in file playback stack.

However for streaming (HLS) playback, you can provide either dvh1 or dvhe and both get recognized. And HLS has better DV support in general because it's the stack used by all streaming apps. So it's actually better if you can distribute your content that can be played by the HLS software stack, so you get equal treatment as all streaming apps today.

You can create Apple conformant HLS streams using the Apple HLS Tools mediafilesegmenter. It's the de-facto authoring app Apple would recommend to streaming content providers for preparing their streaming assets. Of course some advanced vendors like Netflix would use other commercial packaging softwares for the same purpose, but for everyone else, mediafilesegmenter can make sure the output is by the Apple standards.

But playing an HLS asset can be a little bit difficult. Easiest way is to host everything on a local HTTP server and play the HLS m3u8 over HTTP link in Safari.

Distribution of HLS asset can also be challenging since it's a huge folder structure with possibly hundreds of files, and AFAIK currently there's no open standard for packaging HLS assets in a container format. But Apple does have a nice internal format for local HLS asset storage called .movpkg. It's just a folder structure with all the m3u8 and media segment files. Maybe by unofficially documenting this storage format and make it public can help people package and distribute their HLS asset in an universal way.

And the nice thing about .movpkg is that it's locally playable on Apple platform without a local HTTP server, but the player must use AVAssetDownloadURLSession to play them, which is not supported by major player apps today. But if being used, it will utilize the HLS streaming playback stack and give you a better playback experience just like using an streaming app.

Note: Yes AVAssetDownloadURLSession can also download HLS contents and save them to disk under the .movpkg folder structure but it won't do the decryption for you, but nevertheless it's a nice piece of ripping tool built inside Apple's software.

[rixtox]

About file playback stack development, its adaption of newer technology is less active than the streaming stack, as most companies only care about streaming these days. And QuickTime has been deprecated for years already and there won't be any Apple's first-party app to replace it in the near future. It's the world of streaming now. So I would sincerely suggest the WEB-DL scene to do something about this situation and maybe try using the same software stack used by companies providing and playing those contents.

[rixtox]

Having 4, 5, or 6 torrents for a single TV episode just with the distinction of e.g. 720p & 1080p & SDR & HDR10 & DV.mp4 & DV.mkv but the same audio track drives me crazy. They should really start using HLS packaging and let downloader to choose which tier to playback, or even better, let the player to decide and do adaptive streaming from torrent files. Single file container formats should die in the streaming era.

Sorry for my random ranting. I will try to look at the problem and maybe raise issue to Apple but the chance they fix stuff in the file playback stack is next to zero.

[sw5163]

I just had a quick look at HLS, it seems it's very complicated to convert local file to .movpkg, is there currently an open source tool can do this?

For now, I think I will stick with dvh1 by mp4box, most of my files are stored locally or streamed to iOS device through File App from my PC by smb.

I don't think BT group will change to HLS, people will need to learn how to handle a file structure instead of single file. Think about Full BD structure, I always prefer remux.

I will try to look at the problem and maybe raise issue to Apple but the chance they fix stuff in the file playback stack is next to zero.

They won't, they are not even willing to provide TrueHD passthrough. And I spent months to get P8.1 dolby vision to work, just because they deliberately disabled it. Your tool is fine, it's just I saw someone mentioned it on AVSForum, so I thought I should let you know that there is a problem on other Apple devices.

[rixtox]

As I mentioned, mediafilesegmenter is the tool you should use. The output from mediafilesegmenter can be used directly as a .movpkg. The difficult part is to have an app to play it using the correct Apple API.

I'm just hoping someone influencial in the scene can start a change after reading my thoughts, and people would benefit from it.

e.g. making patches to players like VLC to support AVAssetDownloadURLSession with .movpkg format. As long as there's a way to properly play it, people can start distributing it. It's just like the early days with DV, we need the toolings and players to catch up, and there needs to be people pushing the changes.

[sw5163]

Ok thanks! I thought it's only used to make segments and repackage to .movpkg will need other tool, my bad. Will look into it and do some tests.

[sw5163]

我试了一下苹果的这个mediafilesegmenter，文件不在本地播放的话貌似有问题 我把 .m3u8 索引和文件放在服务器上，然后尝试smb打开 .m3u8 要么没反应要么是文本 我不确定是 File App 的 bug 还是我服务器设置有问题

我还试了试先用你的工具转 dvh1 再用 mp4box 重新封装，也黑屏 可能 mp4box 检测到 dvh1 就直接 passthrough 了 最好 readme 说明一下不然 iOS 上转换了以后就只能提取 HEVC 再封装了

[rixtox]

不要用smb服務器，用HTTP服務器，然後用Safari加載M3U8的URL，否則不會啟用HLS player

[sw5163]

Ok 我试试

[sw5163]

有人给我发了这个，你要是有兴趣研究的话

'hvc1' stores all parameter sets inside the MP4 container below the sample description boxes. 
'hev1' stores all parameter sets in band (inside the HEVC stream).

The parameter sets are essentially mandatory metadata necessary to decode the streams.

So you have two slightly different ways of muxing HEVC into MP4.

I'd choose based on the play out and transport scenario.

Your player or your transport specification may require one or the other.

[rixtox]

是的我對比了 mp4box 的輸出也注意到了這一點。我覺得應該有方法不需要重新 muxing 補上那部分的參數，不過還需要做些實驗和測試才行。

[rixtox]

Reference: https://professional.dolby.com/siteassets/content-creation/dolby-vision-for-content-creators/dolby-vision-streams-within-the-http-live-streaming-format-v2.0-13-november-2018.pdf

[rixtox]

Good news: I manually editted a dvhe file to also include the required codec parameters and it works on iPad file playback.

The workflow includes:

1. Merge an segmented MP4 file (multiple moof + mdat boxes) into a continuous MP4 file (single mdat box), and put the moov box after the mdat box. I used ffmpeg -i segmented.mp4 -c copy continuous.mp4 for this job. Probably because of the version of ffmpeg I used, the output changed the sample description entry box from dvhe to hev1, but it's fine since everything inside that sample description entry box seemed to be just copied over identically.

2. Change the dvhe sample description entry box type (due ffmpeg now hev1) to dvh1.

3. Fill in VPS, PPS, and SPS NALUs at the end of the hvcC HEVC configuration box and update other box fields to reflect the change. For this test I copied these NALU entries from the output of mp4box but I believe it can be generated by parsing the HEVC stream.

I'm not sure if making the mdat continuous is really necessary as the main reason I'm doing this is to make manual editing of the moov box easier as that ffmpeg command would put the moov box to the end of the file. The stco Chunk Offsets box references chunks by its file offset. Moving the moov box to the end of the file would make sure the Chunk Offsets remain unchanged while I'm changing the size of the moov box. However I'm not sure if a segmented file can have the moov box moved to the end of file. This need to be tested further.

Anyway I think I can find a time to write some prototype code for this new strategy.

[sw5163]

Sorry I'm really not familiar with all the detailed mp4 file standards. I think mp4muxer is actually using same method as your previous workflow so also leading to black screen on iOS/ATV. But I'm glad you have found a way to fix it. 👍

[sw5163]

It turns out Apple can retrieve the information needed to initialize the decoder even if the file is wrongly muxed. But it requires powerful processor. So on newer iPhone/Mac/ATV playback will not have the black screen problem.

[rixtox]

I'm still planning to address this issue, but it would greatly increase the size of the code because we need to handle full mp4 remux and HEVC bitstream parsing. Not sure when I could have the time to write this though.