search

RenderHeads / UnityPlugin-AVProVideo

AVPro Video is a multi-platform Unity plugin for advanced video playback
https://www.renderheads.com/products/avpro-video/
235 stars 28 forks source link

Videos with DAR show with gaps, and with rotation show as black #627

Closed flchazelas closed 3 years ago

flchazelas commented 3 years ago

The bug When I build WebGL application with 360 video sphere prefab, there is a strip in middle of video. It's probably depend of video size.

Setup :

To Reproduce

  1. Add the 360SphereVideo prefab to the scene
  2. Place the video in StreamingAssets and set the relative path in the MediaPlayer component
  3. Build the project for WebGL

Screenshots Strip

I have tried with a custom sphere setup and had the same issue, but with video (size : 2160x1080) it worked. Do you have a solution?

kahnivore commented 3 years ago

Hi @flchazelas

Not sure offhand why this is happening. Could you please supply the following info:

  1. What is the size and codec of the video that you're having this problem with?
  2. Have you successfully tried this with any of our sample videos?
  3. Have you tried updating your AVPro 1 version to the latest version of this asset (1.11.7)?
  4. What browser are you using for WebGL, and does this problem persist in any others?
  5. Is this problem only apparent in WebGL i.e. not in the editor and not in a Windows build?
flchazelas commented 3 years ago

Hi,

The video size is 2160 x 1200, codec : MP4, and yes, I tried with your sample videos and it work. I just tried with plugin v1.11.0 and with mozilla firefox, google chrome and edge. And It's the same problem. This problem only appears in WebGl, not in editor.

Do you have a solution?

kahnivore commented 3 years ago

We haven't seen this problem before. There's a chance it might have to do with a mismatch between your file's display aspect ratio and source aspect ratio - could you check to see if they match? It might be that WebGL is struggling to resolve this for whatever reason - if this is the case then encoding the videos with the same DAR and SAR would fix the problem.

Let us know if this is the case?

flchazelas commented 3 years ago

There doesn't seem to be a mistake. What do you mean by encoding the same SAR and DAR? In addition, the problem persists with the latest version of your plugin.

kahnivore commented 3 years ago

SAR- source aspect ratio and DAR - display aspect ratio. If your video is encoded with a mismatch, most players will handle this fine but it's possible that this could cause unwanted behaviour.

You're welcome to send your video to unitysupport@renderheads.com and we could take a better look at why this is happening.

kahnivore commented 3 years ago

Have added the original file and a fixed version to S3 bucket for more investigation

AndrewRH commented 3 years ago

TODO: Potentially update the docs if this is a browser issue and we can't fix it in our plugin

stale[bot] commented 3 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

AndrewRH commented 3 years ago

I've made some progress here. Basically the problem is: 1) The source video has a different DAR (display aspect ratio) compared to the aspect ratio of it's frame data, which means the video needs to be scaled to the DAR for display. 2) In webGL/HTML5 there is no way to get information about the DAR 3) In normal HTML5 this is taken care of internally, but because we're copying to a texture, the texture copy needs to also take this into account. 4) It seems like this scaling isn't handled because of the way we're creating our texture. We are creating our own RenderTexture in Unity which the HTML video gets copied to. It could be that this is using texStorage2D() which makes the texture (size etc) immutable...I'm not sure 5) It seems that if we instead use createTexture() inside webgl and then pass that back to Unity as an external texture, then the DAR scaling is applied.

I'm still not 100% sure of the reasons but we appear to have a fix here. This may also fix another issue I discovered which is that videos with rotation metadata aren't transformed by our current method either.

Ongoing investigation...

AndrewRH commented 3 years ago

A side note - if someone is creating a video for mapping to geometry (eg a 360 video) then having DAR scaling is a complete waste of processing power for no benefit.

AndrewRH commented 3 years ago

Found this handy test from Khronos for browsers supporting texture copying from videos with rotation: https://github.com/KhronosGroup/WebGL/blob/master/sdk/tests/conformance/textures/misc/video-rotation.html

AndrewRH commented 3 years ago

This is fixed so I'm closing