google / spatial-media

Specifications and tools for 360º video and spatial audio.
Other
1.86k stars 430 forks source link

YouTube is not detecting the spatial metadata any more #285

Closed JuanIrache closed 1 year ago

JuanIrache commented 1 year ago

Videos with metadata injected with these tools (and others) are not being recognised as VR any more on YouTube. Has something changed recently? I can provide samples privately.

jamiesonpepper commented 1 year ago

Hello! I tested this just now with the tool and a video from my Kodak PixPro 360 and by playing it in VLC and the mouse can pivot around the 360 space without issue.

Here is the same video on YouTube (https://youtu.be/-HgJ38sxfOA) working as well (apologies for the shaky footage it was a small clip that was fast to test, but I don't intend to use).

Are you able to provide an unlisted link or explain what you are seeing further? Unlisted would be the best so I can see exactly what is happening!

JuanIrache commented 1 year ago

Thank you for looking into this.

Interestingly, today some of my test uploads are showing as VR: https://youtu.be/8V3Kf5sqFT8

And some aren't, despite having the same metadata: https://youtu.be/qkE3FcAVAZE

Even if larger videos would take longer to process, uploads I made 24 hours ago are still playing as flat equirectangular.

jamiesonpepper commented 1 year ago

Hmmm that is interesting behavior... Can you run the python spatialmedia <file> command against both files before they were uploaded and paste the results for the working vs non-working ones?

JuanIrache commented 1 year ago

Yes. This one fails:

Loaded file...
        Track 0
        Track 1
                Spherical = true
                Stitched = true
                StitchingSoftware = Spherical Metadata Tool
                ProjectionType = equirectangular

This one shows in VR:

Loaded file...
        Track 0
                Spherical = true
                Stitched = true
                StitchingSoftware = Spherical Metadata Tool
                ProjectionType = equirectangular
        Track 1

The only difference here is the order of streams (audio first vs video first). But I have seen the problem on files with the video first too. The only correlation I have seen is with large vs small videos (or high vs low bitrates). Large files seem to never convert to VR, but it's not entirely consistent. Also, those exact same large files were displayed in VR a couple of months ago

JuanIrache commented 1 year ago

I narrowed this down a bit more. In my workflow, I would normally set the GSpherical stags with Exiftool ( https://exiftool.org/TagNames/XMP.html#GSpherical ). This worked for YouTube until not long ago, still works for VLC.

Now that's not working for YouTube, but if I use the spatial-media tools from this repo to overwrite the tags, the file is not recognised by YouTube either.

If I don't do the Exiftool step and only add tags with this repo, YouTube still recognises it as VR.

JuanIrache commented 1 year ago

Ok. So after comparing files that work on YouTube vs those that don't, in a text editor, the main difference I see is the "XMP" data in files that work is written near the end of the file. Files that don't work have it at the beginning. Also, very small files are interpreted correctly even if the "XMP" data is at the beginning. My only hypothesis is that this is a recently introduced bug on YouTube's end, where the 360 metadata is only looked for within the last N bits of a file or stream. I could be very wrong, of course.

When the spatial-media tool from this repo encounters a file that already has XMP tags, it can overwrite them with new values but it does not move them from the beginning of the file to the end, so the 360 data remains invisible for YouTube.

JuanIrache commented 1 year ago

What would be the best way to report this to get YouTube's attention? I used the "Send feedback" option but I'm not confident that will get to the right team

jamiesonpepper commented 1 year ago

If you can post public Google Drive links to a working and non-working file I'll reach out and see if I can get someone to take a look at this thread and those files.

JuanIrache commented 1 year ago

Here: https://www.dropbox.com/scl/fi/oozbeljc3x6c8jmgpy8z5/clean_exiftool.zip?rlkey=ragpeaqe0iub0u2qfm6u32rff&dl=0

jamiesonpepper commented 1 year ago

Okay thanks! I talked to some folks on the YT encoder side of things and they believe they know what it is and are currently working to fix it. I'll update here when I know more, but it sounds like you have workarounds for now?

JuanIrache commented 1 year ago

Amazing, thanks. I don't have a workaround as I need to do this with exiftool in my workflow, but happy to learn it's being investigated by the relevant team.

nbirkbeck commented 1 year ago

Thanks for the detailed report @JuanIrache.

If you are using FFmpeg in your workflow, you could perhaps also try first exporting the file with "-movflags faststart". Or if you are using the spatialmedial tool, this small patch may serve as a temporary workaround

JuanIrache commented 1 year ago

The ffmpeg movflags did the trick. Thank you @nbirkbeck and @jamiesonpepper ! Is this a temporary fix until YouTube goes back to the previous VR detection method? Or should this be considered permanent?

nbirkbeck commented 1 year ago

Temporary. This was a regression.

jamiesonpepper commented 1 year ago

@JuanIrache Since you have a functional workaround I'm going to close this issue. Thanks again for your report, and the details provided! Thanks @nbirkbeck for the help as well!