search

autonomousvision / kitti360Scripts

This repository contains utility scripts for the KITTI-360 dataset.
MIT License
379 stars 61 forks source link

code for stitching the two fisheyes to make an equirectangular image #27

Closed haruishi43 closed 1 year ago

haruishi43 commented 3 years ago

Thanks for the great dataset!

I know it wasn't this dataset's intention, but I was wondering if it is possible to make a 360 image (equirectangular image) from the two 180 fisheye cameras. If someone did it or if you know any good method to go about this, it would be helpful.

I tried to go do it naively and made a dual-fisheye image and tried to stitch it on the Theta camera's conversion app, but it didn't work out. I haven't really done fisheye conversions, but my guess is that the two fisheyes are too far apart to create a perfect equirectangular image. Then again, it seems like the 180 fisheye seems to be capturing more than 180 degrees (more like 190 degrees?)

Any help would be appreciated.

helmifraser commented 3 years ago

Hi, I've tried tackling this problem myself to limited success.

Going by this answer on Stack Overflow, newer versions of FFmpeg have a v360 filter to convert fisheye/dual fisheye images to equirectangular, which can take either a single image or a video. So I concatenated a fisheye pair to get a dual fisheye image, tested some FOV parameters of the filter (trial and error, really) to get this: v360

Obviously, the sharp border isn't great, which led me down another rabbit hole about blending and saw this open enhancement ticket. Trying a couple of the suggestions in there from Michael Koch (for windows batch, but the arguments for bash are similar enough) got me this: v360-blend

Still isn't ideal, but slightly better than before. I'm sure someone more familiar with blending/stitching can do a much better job than I did. Michael Koch also links his book with some more examples on the filter: http://www.astro-electronic.de/FFmpeg_Book.pdf.

haruishi43 commented 3 years ago

@helmifraser Thanks for sharing with me your workflow. Wow, that looks so much better than mine.

I tried using Hugin to stitch, but can't get reliable control points since it doesn't overlap as much. 0000000000 - 0000000000

I thought it would be easy, but I too fell in the rabbit hole :(

whu-lyh commented 1 year ago

@helmifraser hi helmifraser and @haruishi43 haruishi43 , do you guys know how to convert the fisheye image to an equirectangular image in a batch?