buddhi1980 / mandelbulber2

Official repository for Mandelbulber v2
GNU General Public License v3.0
893 stars 116 forks source link

Alternative camera path interpolation for keyframes #961

Open LivelyLiz opened 6 months ago

LivelyLiz commented 6 months ago

I am one of the authors of the paper "OptFlowCam: A 3D-Image-Flow-Based Metric in Camera Space for Camera Paths in Scenes with Extreme Scale Variations" (see here: https://livelyliz.github.io/OptFlowCam/).

Short summary: we defined a metric that is based on the magnitude of the flow field in the canonical view volume. We were able to get a closed-form solution for the geodesics (or shortest paths) in this space and it yielded good results for the camera path, especially in scenes that have extreme scale differences like space or when zooming in on a fractal.

We actually used Mandelbulber as one of our examples to show how our camera paths differ from the standard linear interpolation, as you can find here https://www.youtube.com/watch?v=_2oiWZafFb8&t=254s

To create that example, we used an external Python script to write a camera path to the Mandelbulber file but I think it would be might be beneficial to implement it in Mandelbulber directly, maybe as an alternative to the standard linear interpolation.

buddhi1980 commented 6 months ago

Thanks for sharing it. The interpolation looks very well. This kind of interpolation would be really beneficial in Mandelbulber. The differences in scale between keyframes sometimes are huge. Estimated distance of the camera to the fractal surface could be used as an input for the interpolation. Would be great if I could see the Python code which you used to create an example animation in Mandelbulber. This way it would be way easier for me to implement it in Mandelbulber (I read the PDF, but math looks a little complicated).

LivelyLiz commented 6 months ago

The code used for interpolation is basically all in this file of our Blender add-on https://github.com/LivelyLiz/OptFlowCam/blob/main/src/optFlowCam/interpolation.py The equations we use there look a bit different than in our paper but are the same when rho=sqrt(2). You could also just use the ones from the paper instead.

The rest of the script was mainly just reading/writing the Mandelbulber file and converting camera models. I can upload the script files we used to a gist or this issue if you need the whole thing. The code for that is a bit messy because it was originally just meant to be used for a quick proof-of-concept but was good enough for generating an example :D