trek-view / gopro2gsv

Processes .jpg images and .mp4/.360 videos shot on GoPro MAX or Fusion cameras and uploads to Google Street View.
Apache License 2.0
4 stars 0 forks source link

Change way video->video processing pipeline works #26

Closed himynamesdave closed 9 months ago

himynamesdave commented 9 months ago

The video->video pipeline does not modify the video at present, except for allowing user to add nadir https://github.com/trek-view/gopro2gsv#video---video-processing-pipeline

It should be possible for a user to turn a video into images first, that can then be rebuilt to a video.

This has 2 benefits

  1. reduces output video size, as normal video is 27 FPS, (e.g. user might reduce to 1 FPS -- a 26x reduction).
  2. allows for outliers to be removed, as per #25

For this, in video->video mode a user should select an --extract_fps flag (a number in dictionary)

User can enter values between 10, 8, 5, 4, 2, 1, 0.5 (1 frame / 2 seconds), 0.2 (1 frame / 5 seconds).

The logic to break up mp4 videos (and add geotags) can be found here: https://github.com/trek-view/gopro2frames

Note, the code from this repo can be used, but only the code relevant to converting MP4 to images (with geotags) should be used. It might also need some slight modification to improve accuracy of GPS points assigned to photos.

Once a series of geotagged images results from the video, they will then be processed as per the image->video pipeline including the new function; #25

https://github.com/trek-view/gopro2gsv#image---video-processing-pipeline

In this mode, a user should be able to set --keep_extracted_frames which will also store a copy of the extracted images used to create final video (with geotags). If not set, frames extracted will be removed on completion.

fqrious commented 9 months ago

Is --keep-extracted-frames actually needed, I believe it's better to do most of the operations in memory such that there isn't a need to geotag every single frame before running the image-video pipeline on it

himynamesdave commented 9 months ago

@fqrious , yes, it is needed. The reason it is optional is because sometimes a user wants to keep the geotagged frames for other purposes, outside of gopro2gsv

fqrious commented 9 months ago

Alright, I'll do that...

The problem here is that there is significantly less number of gps data than there are frames, it's like 1:4 ratio... I already sorted it out anyways, I just wanted to confirm

fqrious commented 9 months ago

If extract_fps isn't set, should it default to 10 or video_fps?

himynamesdave commented 9 months ago

@fqrious if no extract_fps value set, then we should process the video as we do now. That is, do not break it into frames and pass it on as it is.

fqrious commented 9 months ago

done

himynamesdave commented 9 months ago

Closing this as done, will raise individual issues individually.