Android VR Video freezes for several seconds on initial playback from internal storage, Runs fine if played immediately after downloading to quest 2 storage via UnityWebRequest

[TomDdotExe]

Please DO NOT LINK / ATTACH YOUR PROJECT FILES HERE

Hey There,

I'm currently developing a VR video player that includes no video files in the initial APK, instead downloading the files to Application.persistentDataPath on the first time accessing a video, with the goal of then being able to load from internal storage on subsequent app playthroughs.

On the first run after apk installation, I'm able to download the .mp4 file and play it back with no frame drops/freezing.

The issue I'm encountering is if I close the app, reopen and then attempt to play the local file, it plays back initially at a low frame rate and completely freezes for several seconds, before unfreezing and proceeding to play without issue going forward.

I don't suppose this has a fairly straightforward solution? My newb suspicion is that the local file needs to be pre-cached before playback, so I'm hoping that this is just a case of someone confirming that's the case.

Thanks

• Unity version: 2021.3.18f1
• AVPro Video version (number and edition (trial/core/ultra/enterprise)): Trial
• Operating system version: W11
• Device model: Quest 2
• Video specs (resolution, frame-rate, codec, file size): 5760x5760, 29.97, H265, 570MB.

[Ste-RH]

That’s a pretty big video. Check the moov atom is at the start of your video file. If not, you can do this by re-encoding with ffmpeg and include the flag: -movflags +faststart

If you were using the Ultra Edition then you could use the download/caching feature instead to going to the trouble of downloading the video yourself. This has the added benefit of being able to playback whilst downloading the video to the local file system, as well as obfuscating the video file so users cannot access it outside your app.

[TomDdotExe]

Thanks for the response,

I've just added fast start to the mp4 using FFmpeg (which resulted in the file size increasing to 713mb) and ironically the video now stutters even worse when ran locally off of the quest 2 haha. The video still runs fine after immediate download, just after app reboot the stuttering starts, and with fast start enabled doesn't seem to stop.

Thanks

[Ste-RH]

The size increase will be down to maybe a codec change? Or bitrate. One of those. Ensure you are marching. The moov atom is tiny and a constant size. Ensure you are setting your re-encode to be with HEVC if you are not.

What is the bitrate of your original video? That might be playing a part here.

[TomDdotExe]

Thanks again for your response. I had a look into the FFmpeg commands I was using to add the moov atom, and ended up using the following command to add it without the previous file size increase

ffmpeg -i CombinedVidAndAudio.mp4 -c copy -map 0 -movflags +faststart out.mp4

Unfortunately, the issue still persists only on the local file read. so I've kept moving on with other solutions

I had a look into the file bitrate, which I have to be honest as someone quite new to VR Video I'm not currently aware of the best way to identify the bitrate of a video - Windows properties says the videos Data Rate is 12687kbps, and the Total Bitrate is 12941kbps, with the codec being (MPEG-H Part2/HEVC (H.265) (hev1).

If it helps at all, while looking for alternative ways to find the video's bitrate, I located the bitrate graph in VLC player. I noticed that the graph spikes at the start of the video at the exact moment the in-headset freezing begins.

[TomDdotExe]

So I've found an app called MediaInfo which allows me to provide a bit more specific information about the video file. I'll paste the entire thing here as not sure which parts are useful for diagnostics.

General Complete name: C:\ffmpeg\CombinedVidAndAudio.mp4 Format : MPEG-4 Format profile: Base Media Codec ID: isom (isom/iso2/mp41) File size: 565 MiB Duration: 6 min 6 s Overall bit rate: 12.9 Mb/s Writing application: Lavf59.27.100

Video ID: 1 Format: HEVC Format/Info: High Efficiency Video Coding Format profile: Main@L6@Main Codec ID: hev1 Codec ID/Info: High Efficiency Video Coding Duration: 6 min 6 s Bit rate: 12.7 Mb/s Width: 5 760 pixels Height: 5 760 pixels Display aspect ratio: 1.000 Frame rate mode: Constant Frame rate: 29.970 (30000/1001) FPS Color space: YUV Chroma subsampling: 4:2:0 Bit depth: 8 bits Bits/(Pixel*Frame): 0.013 Stream size: 554 MiB (98%) Writing library : x265 3.1.2+1-76650bab70f9:[Linux][GCC 4.8.5][64 bit] 8bit Encoding settings: cpuid=1111039 / frame-threads=2 / wpp / no-pmode / no-pme / no-psnr / no-ssim / log-level=2 / input-csp=1 / input-res=5760x5760 / interlace=0 / total-frames=0 / level-idc=0 / high-tier=1 / uhd-bd=0 / ref=3 / no-allow-non-conformance / repeat-headers / annexb / no-aud / no-hrd / info / hash=0 / no-temporal-layers / open-gop / min-keyint=25 / keyint=250 / gop-lookahead=0 / bframes=4 / b-adapt=2 / b-pyramid / bframe-bias=0 / rc-lookahead=20 / lookahead-slices=8 / scenecut=40 / radl=0 / no-splice / no-intra-refresh / ctu=64 / min-cu-size=8 / no-rect / no-amp / max-tu-size=32 / tu-inter-depth=1 / tu-intra-depth=1 / limit-tu=0 / rdoq-level=0 / dynamic-rd=0.00 / no-ssim-rd / signhide / no-tskip / nr-intra=0 / nr-inter=0 / no-constrained-intra / strong-intra-smoothing / max-merge=3 / limit-refs=1 / no-limit-modes / me=1 / subme=2 / merange=57 / temporal-mvp / weightp / no-weightb / no-analyze-src-pics / deblock=0:0 / sao / no-sao-non-deblock / rd=3 / early-skip / rskip / no-fast-intra / no-tskip-fast / no-cu-lossless / b-intra / no-splitrd-skip / rdpenalty=0 / psy-rd=2.00 / psy-rdoq=0.00 / no-rd-refine / no-lossless / cbqpoffs=0 / crqpoffs=0 / rc=crf / crf=23.0 / qcomp=0.60 / qpstep=4 / stats-write=0 / stats-read=0 / ipratio=1.40 / pbratio=1.30 / aq-mode=2 / aq-strength=1.00 / cutree / zone-count=0 / no-strict-cbr / qg-size=32 / no-rc-grain / qpmax=69 / qpmin=0 / no-const-vbv / sar=0 / overscan=0 / videoformat=5 / range=0 / colorprim=2 / transfer=2 / colormatrix=2 / chromaloc=0 / display-window=0cll=0,0 / min-luma=0 / max-luma=255 / log2-max-poc-lsb=8 / vui-timing-info / vui-hrd-info / slices=1 / no-opt-qp-pps / no-opt-ref-list-length-pps / no-multi-pass-opt-rps / scenecut-bias=0.05 / no-opt-cu-delta-qp / no-aq-motion / no-hdr / no-hdr-opt / no-dhdr10-opt / no-idr-recovery-sei / analysis-reuse-level=5 / scale-factor=0 / refine-intra=0 / refine-inter=0 / refine-mv=0 / refine-ctu-distortion=0 / no-limit-sao / ctu-info=0 / no-lowpass-dct / refine-analysis-type=0 / copy-pic=1 / max-ausize-factor=1.0 / no-dynamic-refine / no-single-sei / no-hevc-aq / no-svt / no-field / qp-adaptation-range=1.00 Color range: Limited Codec configuration box: hvcC

Audio ID: 2 Format: AAC LC Format/Info: Advanced Audio Codec Low Complexity Codec ID: mp4a-40-2 Duration: 6 min 6 s Bit rate mode: Constant Bit rate: 253 kb/s Channel(s): 2 channels Channel layout: L R Sampling rate: 48.0 kHz Frame rate: 46.875 FPS (1024 SPF) Compression mode: Lossy Stream size: 11.1 MiB (2%) Default: Yes Alternate group: 1

Thanks again for the help!

[Chris-RH]

The point where the bitrate spikes, what is the reading before the spike and at the highest point of the spike?

[TomDdotExe]

Pre-spike the video input bitrate is approximately 7000kb/s, the Spike is approx 120,000kb/s, and post-spike the bitrate settles at approx 20,000kb/s.

Thanks again.

[Ste-RH]

120mbit/sec is quite high even for the Quest 2. As a test, do a full re-encode and bring the bitrate down to something a lot lower. You will need to specify a codec and a rate control. This link will help:

https://trac.ffmpeg.org/wiki/Encode/H.265

Get the bitrate down to 50 or lower across the whole video and you will hopefully see an improvement in your issue.

[Chris-RH]

Bitrate: Recommended max 100Mbps (https://creator.oculus.com/getting-started/media-production-specifications-for-delivery-to-meta-quest-2-headsets/)

[TomDdotExe]

Hey again,

I've spent this evening reencoding the video to be under 50mbps. However, the issue still persists entirely the same as with the original file - the downloaded file runs without any performance hitches (even the version that spikes at 120mbps), only to start freezing on the second launch and attempted playback from local storage.

The only other bit of information I can currently think could be helpful is that I'm currently downloading the file to Application.persistentDataPath.

The video performs without frame drops if included in streamingAssets on APK build, but from what I can gather there is no way to initially download a file to PersistentData and then move it over to StreamingAssets for future playback, so isn't of much help in this instance.

Thanks Again

[Ste-RH]

Is it possible to email over a demo project to us so we can replicate it here? You can send it over to unitysupport@renderheads.com

[Chris-RH]

do you still need help with this?