[request] Meshroom for Focus stacking?

[hcw70]

I was thinking if Meshroom can be used to create a 3D model from a set of focus stacking images. By focus stacking i mean macro photography where i.e. an insect is shoot with shallow depth of field with many images (50-200) while moving the camera towards the insect. So different depth slices become in focus during the sequence.

"Normal" focus stacking software (Helicon Focus / Zerena stacker) create a plain 2d image, (but can produce a 3D depth map based on the image alignment and the sharpness estimation).

Is it possible to setup a pipeline in Meshroom which creates a 3D model from a focus stack?

Roughly processing:

• Find in-focus parts of each image (Laplacian / Sobel filter)
• align consecutive images based on in-focus parts (SIFT?)
• Get Camera motion based on alignment
• Build a mesh around sharp points of consecutive images
• simplify mesh
• add in-sharp pixels from each image onto mesh at best-fit locations

See https://stackoverflow.com/questions/15911783/what-are-some-common-focus-stacking-algorithms

[fabiencastan]

Hi, Could you contact me at fabien.castan@mikrosimage.com to setup a confcall to discuss about it?

[natowi]

Here are two links on the topic:

https://www.reddit.com/r/photogrammetry/comments/anyr4s/focus_stacking_with_an_automated_turntable/

Perspective-consistent multifocus multiview 3D reconstruction of small objects https://arxiv.org/pdf/1912.03005.pdf

The option to generate a depth map from the image stack is really interesting. Here is a nice demonstration: https://github.com/rAum/FocusStacking (pre-compiled binaries)

[hcw70]

@fabiencastan Lets discuss.. I did sent you a mail....

@natowi Thanks for the links. Interresting paper.... 8->

[pepae]

Anything happened in this regard? It would be amazing for macro photogrammetry workflows if Meshroom did the focus stacking automatically

[natowi]

@pepae You could use existing tools like https://github.com/PetteriAimonen/focus-stack or https://pypi.org/project/focus-stack/ and write a new Meshroom node to call a stacking tool. (Similar to this) The input should ideally be one folder with one subfolder per stack of images (at the moment, all the tools I know of, read all the images from on folder only and output one stacked image) otherwise one could define a batch number of images for stacking. The output should be multiple stacked images in one folder. (Example node: https://github.com/natowi/meshroom_external_plugins/blob/master/InstantMeshes.py) Documentation https://meshroom-manual.readthedocs.io/en/latest/feature-documentation/core/nodes.html Additional notes: https://github.com/natowi/meshroom_external_plugins/wiki/2-DevDoc