Make use of automatic segmentation to delineate objects of interest

[SamuelJoutard]

Hi,

I would like to make use of automatically segmented images in order to post-process the mesh obtained efficiently (i.e. remove most of what needs to be removed automatically).

The idea is fairly simple:

1. I take pictures of a body
2. I can obtain good 2D segmentations of that body using DeeplabV2 for instance (they are not perfect but I would use this as a first step).
3. I use meshroom with my RGB images to construct a 3D mesh of the scene (including the body).
4. I filter out part of the mesh that is not relevant for me using the 2D segmentations.

One way to do this last step would be for instance to colour the mesh using the 2D segmentations binarized (black for the background, white for the foreground) and filter vertices based on their colour.

I did not manage to obtain satisfying results so far. I tried to play with the CLI plugging the binary segmentation at the texturing step but it did not work.

It would be super helpful if I could get some hint about how to do that.

Thanks

Samuel

[natowi]

Linking to https://github.com/alicevision/meshroom/issues/566

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.