search

RenderHeads / UnityPlugin-AVProVideo

AVPro Video is a multi-platform Unity plugin for advanced video playback
https://www.renderheads.com/products/avpro-video/
224 stars 27 forks source link

Videos larger than 5M will fail to play, while videos smaller than 5M will play normally #1814

Closed KeeleyYu closed 2 months ago

KeeleyYu commented 2 months ago

Unity version: 2020.3.25f1 AVPro Video version (number and edition (trial/core/ultra/enterprise)): ultra Operating system version: v2.7.2 Device model: IOS Video specs (resolution, frame-rate, codec, file size): none

History related issue: #1646

As mentioned in the issue above, which is 'with the same encoding (h264), videos larger than 5M will fail to play, while videos smaller than 5M will play normally', we also did a lot of testing and found that the following devices have the same problem (mainly happens on IOS) : iPhone X、iphone 11、iphone12、iPad 6 and more lower-end devices

Could you please help?

KeeleyYu commented 2 months ago

More specifically, the expected and correct logs on Iphone14 are as follows: MetaDataReady, ResolutionChanged, ReadyToPlay, FirstFrameReady

The incorrect logs on Iphone11 and below are as follows: MetaDataReady, ResolutionChanged

We started Play after receiving FirstFrameReady, so the abnormal behavior now is that the screen remains black.

Chris-RH commented 2 months ago
  1. Do they run on those devices using their native video players?
  2. What devices are the videos able to run on?
  3. Please update to the latest version of AVPro Video 2.9.3 and retest. (Please note that AVPro video version 2 has been depreciated and will no longer be receiving updates)
  4. Can you reproduce this in a new project running only AVPro Video mediaplayer demo scene and your videos?
  5. Are they playing locally or streaming?
  6. What are the video specs? (resolution, codec, FPS, bitrate etc)
  7. Please you supply a sample of the video you are using for testing to unitysupport@renderheads.com
  8. Please can you supply a full error log
KeeleyYu commented 2 months ago

@Chris-RH

  1. I don’t understand the definition of native. If it refers to whether the video can be played from the system player, then it is ok.

  2. Iphone13,14 and above include pro version devices

  3. The game is currently in operation. If it needs to be upgraded, it will cost a certain amount of money to conduct sufficient testing, so we want to solve this problem at the minimum cost. Of course I will also try to upgrade locally and retest

  4. I will try it

  5. locally

  6. The information output through ffmpeg -i is as shown in the figure image

  7. Sorry, we cannot provide video resources due to confidentiality agreements, but we have tried some other public resources, and the same problem will occur as long as it is larger than 5M, such as the classic demo big buck bunny, you can try it

  8. The device logs with problems are as shown in the first figure, and the device logs with no problems are as shown in the second figure, while avpro has no more other error messages output. image image

MorrisRH commented 2 months ago

We regularly test videos that are serveral 100Mb in size so this feels like an issue with the encoding. Without access to the source videos this will be tough to diagnose.

Could you try passing one of the videos that fails through the following ffmpeg command to see if that fixes the issue?

ffmpeg -i video.mp4 -c:v copy -c:a copy -write_tmcd 0 -movflags faststart out.mp4
KeeleyYu commented 2 months ago

We regularly test videos that are serveral 100Mb in size so this feels like an issue with the encoding. Without access to the source videos this will be tough to diagnose.

Could you try passing one of the videos that fails through the following ffmpeg command to see if that fixes the issue?

ffmpeg -i video.mp4 -c:v copy -c:a copy -write_tmcd 0 -movflags faststart out.mp4

It works! Great. Thank you! But I compared these two videos and could not see much obvious difference. Can you please explain? @MorrisRH

Chris-RH commented 2 months ago

What it does is to relocate the moov atom from the end, to the beginning of the video. This means that the file will start playing straight away instead of waiting until it's fully downloaded / buffered.

The moov atom defines the timescale, duration, display characteristics etc of the video. Because the moov atom uses absolute file sizes in its format, putting at the beginning of the file before the entire file has been created isn’t possible. The only way to move it to the beginning is to generate the entire file with the moov atom at the end, and then re-process the entire file to move it to the beginning.

MorrisRH commented 2 months ago

We regularly test videos that are serveral 100Mb in size so this feels like an issue with the encoding. Without access to the source videos this will be tough to diagnose.

Could you try passing one of the videos that fails through the following ffmpeg command to see if that fixes the issue?


ffmpeg -i video.mp4 -c:v copy -c:a copy -write_tmcd 0 -movflags faststart out.mp4

It works! Great. Thank you!

But I compared these two videos and could not see much obvious difference. Can you please explain?

@MorrisRH

My hunch is that in your videos the audio and video tracks weren't interleaved, but two separate blocks so the decoder naively keeps decoding video frames whilst looking for audio frames.

Running the video through the above ffmpeg command just interleaves them without re-encoding.

You can use mp4info from the bento4 tools to easily view the sample layout.

KeeleyYu commented 2 months ago

Oh I see. Thanks a lot for your patient explanation. One last question: For the same video that has not been re-encoded, why can high-end devices successfully play it while others can not? Are there any pitfalls in the iOS system?

MorrisRH commented 2 months ago

Oh I see. Thanks a lot for your patient explanation. One last question: For the same video that has not been re-encoded, why can high-end devices successfully play it while others can not? Are there any pitfalls in the iOS system?

I expect it comes down to the amount of available memory the device has. The more recent devices come with more RAM than older devices and are therefore able to hold more decoded frames in memory.

KeeleyYu commented 2 months ago

Sorry for bother again. I used mp4info from bento4 to output the information of the two videos before and after encoded. As shown as follows. image image

The fast start fields are all 'yes', which I think represents the location of the moov atom. So does it mean that the problem of the video being unable to play before encoded is not actually caused by moov atom? For reference, I also used the mp4box -info command to show some information which are as follows. image image

@MorrisRH

MorrisRH commented 1 month ago

Use the show layout option with mp4info like this:

mp4info --show-layout file.mp4

It will show the sample layout after the file and track info and it should look something like the following with the video ([V]) and audio ([A]) samples interleaved:

00000000 [V] (1)* size= 21624, offset=   16337, dts=0 (0 ms)
  00000001 [V] (1)  size=   838, offset=   37961, dts=1000 (40 ms)
  00000002 [V] (1)  size=   575, offset=   38799, dts=2000 (80 ms)
  00000003 [V] (1)  size=  6523, offset=   39374, dts=3000 (120 ms)
  00000004 [V] (1)  size=  3419, offset=   45897, dts=4000 (160 ms)
  00000005 [V] (1)  size=  3835, offset=   49316, dts=5000 (200 ms)
  00000006 [V] (1)  size=  2382, offset=   53151, dts=6000 (240 ms)
  00000007 [V] (1)  size= 14020, offset=   55533, dts=7000 (280 ms)
  00000008 [V] (1)  size=  9271, offset=   69553, dts=8000 (320 ms)
  00000009 [V] (1)  size=  9718, offset=   78824, dts=9000 (360 ms)
| 00000010 [A] (2)* size=    10, offset=   88542, dts=0 (0 ms)
| 00000011 [A] (2)* size=    10, offset=   88552, dts=1024 (21 ms)
| 00000012 [A] (2)* size=   473, offset=   88562, dts=2048 (43 ms)
| 00000013 [A] (2)* size=   505, offset=   89035, dts=3072 (64 ms)
| 00000014 [A] (2)* size=   506, offset=   89540, dts=4096 (85 ms)
| 00000015 [A] (2)* size=   504, offset=   90046, dts=5120 (107 ms)
| 00000016 [A] (2)* size=   506, offset=   90550, dts=6144 (128 ms)
| 00000017 [A] (2)* size=   504, offset=   91056, dts=7168 (149 ms)
| 00000018 [A] (2)* size=   506, offset=   91560, dts=8192 (171 ms)
| 00000019 [A] (2)* size=   504, offset=   92066, dts=9216 (192 ms)
| 00000020 [A] (2)* size=   506, offset=   92570, dts=10240 (213 ms)
| 00000021 [A] (2)* size=   504, offset=   93076, dts=11264 (235 ms)
| 00000022 [A] (2)* size=   506, offset=   93580, dts=12288 (256 ms)
| 00000023 [A] (2)* size=   504, offset=   94086, dts=13312 (277 ms)
| 00000024 [A] (2)* size=   506, offset=   94590, dts=14336 (299 ms)
| 00000025 [A] (2)* size=   504, offset=   95096, dts=15360 (320 ms)
| 00000026 [A] (2)* size=   506, offset=   95600, dts=16384 (341 ms)
| 00000027 [A] (2)* size=   504, offset=   96106, dts=17408 (363 ms)
| 00000028 [A] (2)* size=   506, offset=   96610, dts=18432 (384 ms)
  00000029 [V] (1)  size= 19683, offset=   97116, dts=10000 (400 ms)
  00000030 [V] (1)  size=  4431, offset=  116799, dts=11000 (440 ms)
  00000031 [V] (1)  size= 16467, offset=  121230, dts=12000 (480 ms)
  00000032 [V] (1)  size=  3550, offset=  137697, dts=13000 (520 ms)
  00000033 [V] (1)  size= 11811, offset=  141247, dts=14000 (560 ms)
  00000034 [V] (1)  size= 11415, offset=  153058, dts=15000 (600 ms)
  00000035 [V] (1)  size=  3269, offset=  164473, dts=16000 (640 ms)
  00000036 [V] (1)  size=  9316, offset=  167742, dts=17000 (680 ms)
  00000037 [V] (1)  size=  7910, offset=  177058, dts=18000 (720 ms)
  00000038 [V] (1)  size=  3455, offset=  184968, dts=19000 (760 ms)
| 00000039 [A] (2)* size=   627, offset=  188423, dts=19456 (405 ms)
| 00000040 [A] (2)* size=   621, offset=  189050, dts=20480 (427 ms)
| 00000041 [A] (2)* size=   643, offset=  189671, dts=21504 (448 ms)
| 00000042 [A] (2)* size=   750, offset=  190314, dts=22528 (469 ms)
| 00000043 [A] (2)* size=   697, offset=  191064, dts=23552 (491 ms)
| 00000044 [A] (2)* size=   565, offset=  191761, dts=24576 (512 ms)
| 00000045 [A] (2)* size=   470, offset=  192326, dts=25600 (533 ms)
| 00000046 [A] (2)* size=   588, offset=  192796, dts=26624 (555 ms)
| 00000047 [A] (2)* size=   498, offset=  193384, dts=27648 (576 ms)
| 00000048 [A] (2)* size=   435, offset=  193882, dts=28672 (597 ms)
| 00000049 [A] (2)* size=   495, offset=  194317, dts=29696 (619 ms)
| 00000050 [A] (2)* size=   496, offset=  194812, dts=30720 (640 ms)
| 00000051 [A] (2)* size=   525, offset=  195308, dts=31744 (661 ms)
| 00000052 [A] (2)* size=   498, offset=  195833, dts=32768 (683 ms)
| 00000053 [A] (2)* size=   489, offset=  196331, dts=33792 (704 ms)
| 00000054 [A] (2)* size=   598, offset=  196820, dts=34816 (725 ms)
| 00000055 [A] (2)* size=   458, offset=  197418, dts=35840 (747 ms)
| 00000056 [A] (2)* size=   561, offset=  197876, dts=36864 (768 ms)
| 00000057 [A] (2)* size=   506, offset=  198437, dts=37888 (789 ms)

If all the audio or video samples are in a single block (not-interleaved) then that is the problem.