trek-view / gopro2gsv

Processes .jpg images and .mp4/.360 videos shot on GoPro MAX or Fusion cameras and uploads to Google Street View.
Apache License 2.0
4 stars 0 forks source link

Issue with .360 output aspect ratio #52

Closed himynamesdave closed 9 months ago

himynamesdave commented 9 months ago

Original spec (below under =====) was to use max2sphere.

We now use ffmpeg #48 (which is fine in theory).

However, the aspect ratio created from the frames is incorrect and not accepted by GSV

image

Can we resolve this with ffmpeg?

If not, we might have to go back to max2sphere.

https://github.com/trek-view/max2sphere/blob/main/docs/dev-notes.pdf (here output is dependant on input size, and uses custom masks)

Note, using ffmpeg to force aspect ration changes is not acceptable, as it will distort image

======

Processing is broken up into four key parts;

  1. extract GPS from video
  2. extracting frames from .360 input
  3. turning extracted frames into equirectangular projections using max2sphere
  4. geotag extracted frames
4c Processing - extract GPS from video

Same as 3c: Processing - extract GPS

4c Processing - extracting frames from .360 input

The following command can be used to extract frames from the video;

ffmpeg -i INPUT.360 -map 0:0 -r XXX trackN/img%d.jpg -map 0:5 -r XXX trackN/img%d.jpg

Where XXX = framerate user passes in CLI.

Note: if timewarp mode is used, the track numbers are different:

Regular video = -map 0:0 and -map 0:5 Timewarp video = -map 0:0 and -map 0:4 (not currently supported)

4c Processing - turning extracted frames into equirectangular projections using max2sphere

Using max2sphere each frame can be turned into equirectangular frames;

@SYSTEM_PATH/max2sphere -w XXXX -n 1 -m YYYY track%d/frame%4d.jpg

Note, -w flag variable (XXXX), is the same as ImageWidth of extracted frame:

For -m flag variable YYYY is equal to number of frames extracted from the video.

4c Processing - geotag extracted frames

Same as 3c: Processing - geotag the extracted frames.

Script now continues rest 2c processing (GPS smoothing)

fqrious commented 9 months ago

I can just scale with ffmpeg, I don't want to deal with compiling an external library on install or whatever

himynamesdave commented 9 months ago

@fqrious that's not acceptable, as quality will be lost.

Did you see https://github.com/trek-view/gopro2frames

This is already implemented there

fqrious commented 9 months ago

Well, if you insist.

himynamesdave commented 9 months ago

@fqrious this is still happening

https://github.com/trek-view/max2sphere#examples-macos

Have a read of that.

You need to read the input video size (it can be one of two sizes) and apply the correct -w flag depending on the input