google / spatial-media

Specifications and tools for 360º video and spatial audio.
Other
1.85k stars 428 forks source link

Youtube does not accept custom mesh. #210

Open BorisBogaerts opened 5 years ago

BorisBogaerts commented 5 years ago

Hi,

I am trying to develop a custom mesh format (VR180 format) to upload to youtube. It is similar to a cube map but only for a half cube (better resolution). I've added this mesh using the tools in this repository. I've debugged it to the point where I know that triangle winding is correct (both on mesh and texture), the triangle strip is correctly defined, etc. When I upload the video to youtube, it states that the vr format is unknown.

youtube error

It is clear that youtube rejects the custom mesh based on some unspecified criterion. Does anyone have any idea what this could be? Any help would be greatly appreciated. Or does anyone even know if Youtube supports the complete VR180 format (or only partially)?

Following is standard (minimal) fish eye mesh using the tool that is working (on youtube). working

Following is the custom mesh that is not working (on youtube) notworking

Blue vector is -Z axis, red is +X and green is +Y. Red vectors on each triangle indicates triangle normals (all point outward). Green triangles on texture all have the same winding order (otherwise the color would be red). The visualization correctly unwinds triangles, draws all triangles and all vertices based on the specification on wikipedia: https://en.wikipedia.org/wiki/Triangle_strip

Command arguments "args": [ "-i", "-s" , "left-right", "-m", "half-cube", //"mesh", "testVR180_newFormat2.mp4", "upload.mp4" ] Here "half-cube" is self defined and exactly the same as "mesh", the only difference is that it calls a different mesh generation function in mesh.py. This was validated, if the standart mesh generation code is used, the video is recognized by youtube.

Vargol commented 5 years ago

Hi.

I think you raised this in the wrong fork the one doesn't do mesh projection.

Youtube definitely do some 'validation' on the meshes, as I've had plenty of failed experiments along similar lines. This is what I learnt....

YT wants an OpenGL compliant mesh, so the strip requires degenerate triangle to wrap around. There appears to some UV checks going on it seems happy for a hemi-sphere to start at roughly <0 , 0.5> but as soon as you start to flatten stuff out the mesh it seems to want values closer to <0,1> <0,0> <0,1> <1,1> at the corners.

I didn't get multiple strips working, but possibly never tried it with with correct mesh generation, although it is part of the standard. So if you using a strip per cube face that might be an issue

Having said that, what I'd try, assuming you haven't already, is making your mesh fully connected. For example, style it on the minimal fisheye with an extra ring of vertices in it and the centre vertex level with that extra line, so the the sides of the open cube are the traditional square of two triangles but the bottom is a square of four triangles pointing to the centre.

Other than that I'd go for a mesh based on a T-shaped net with the degenerate triangles.

oh, one other thing, you might want to try changing the mshp box type label to ytmp which is something I've seem YT do when experimenting with cube mapping, indeed for the first few months I had with VR180 mesh projection working YT was not accepting the official box name.

BorisBogaerts commented 5 years ago

Hi Vargol,

Thanks for the comment and suggestions.

The mesh I used is one single triangle strip (so it contains degenerate triangles). I experimented with multiple separate strips (or even separate triangles which is in theory supported) and nothing I tried got accepted.

I also tried the thing you mentioned with the fisheye mesh with an extra ring and this did not work.

I quickly replaced the mshp label to ytmp and this also did not work.

The problem with experimenting with a custom mesh is that I do not have enough accepted attempts which makes reverse engineering impossible. So for now I abandoned this project.