[BUG] 360 video is flattened

[JesseWebDotCom]

The bug

When re-encoding 360 video, immich flattens the video.

To confirm:

• take a 360 degree mp4 video, open in VLC, and confirm you can properly rotate the video in 360 degrees
• now have immich re-encode that video (HEVC in my case) and do the same test in VLC - you will not be able to rotate the video

My theory is immich loses the 360 side data when using FFmpeg to re-encode. For example, here is some ffprobe side data for a 360 video that I believe immich loses:

  Stream #0:0[0x1](und): Video: hevc (Main) (hvc1 / 0x31637668), yuv420p(tv, bt709), 4096x2048 [SAR 1:1 DAR 2:1], 28334 kb/s, 29.97 fps, 29.97 tbr, 90k tbn (default)
    Metadata:
      creation_time   : 2022-07-27T13:27:56.000000Z
      handler_name    : VideoHandler
      vendor_id       : [0][0][0][0]
    Side data:
      spherical: equirectangular (0.000000/0.000000/0.000000) 

When re-encoding, immich needs to add -strict unofficial flag to the ffmpeg command line in order to copy the video projection metadata. See here for more detail: https://stackoverflow.com/questions/44760588/preserving-side-data-information-for-360-video-transcoding-using-ffmpeg/48147865#48147865

The OS that Immich Server is running on

Ubuntu 22.10

Version of Immich Server

v1.89.0

Version of Immich Mobile App

v1.89.0

Platform with the issue

• [X] Server
• [X] Web
• [X] Mobile

Your docker-compose.yml content

n/a

Your .env content

n/a

Reproduction steps

see above

Additional information

No response

[johmicrot]

I'm on the latest release (v.1.111.0) and this is still a persisting issue

[mertalev]

Sorry, we have to revert the PR for this for now. It had the side effect of breaking playback when the source video uses Dolby Vision. The transcode ends up having incompatible DOVI metadata that players try and fail to use. It's surprisingly difficult to remove this and just this metadata, but will be easier in FFmpeg 7.1.