Image masking

[robertguetzkow]

I'd like to suggest image masking as a feature for a future version of Meshroom. In case this would be implemented, masked areas should not be used for keypoint detection nor for depth maps and texture generation. A simple import of masks in the form of binary images would be enough, a build in editor is not necessary.

Thank you for developing this great photogrammetry software!

[endolith]

Because the sensor information does not match the image resolution. Removing the metadata results in Meshroom estimating the parmeters.

Ok, I'll try it. It doesn't detect a conflict between metadata and resolution and automatically switch to estimation?

Good to know, thank you. @endolith you could give this a try

I tried white, too, but it was basically the same:

Though in some parts it did work around the "mask", so maybe the distorted parts just didn't have enough exposures to combine:

[natowi]

It doesn't detect a conflict between metadata and resolution and automatically switch to estimation?

Simply check how many cameras are defined in the cameraInit node

Though in some parts it did work around the "mask"

The mask works in areas that are covered by multiple other images, therefore the masked area will be marked as outlier and removed from the reconstruction. If you mask out the area in multiple images, this is no longer possible and due to the lack of features to match, the area will be left out. (Meshroom will be unsure whether the white area is supposed to be there or not) This is a similar behaviour to moving objects in your image.

In your case I would not create squared masks that partially cover the stone. I would only remove/mask out the background of the image, ignoring parts that overlay the stone. If you have enough images, Meshroom will ignore the areas that you wanted to mask out initially.

[TheConceptBoy]

has there been any developments in this area?

[eyast]

@TheConceptBoy - I dont know. I came here accidentally. I've personally played with rembg on pre (https://github.com/danielgatis/rembg) and I am getting better results

[addohm]

@TheConceptBoy - I dont know. I came here accidentally. I've personally played with rembg on pre (https://github.com/danielgatis/rembg) and I am getting better results

This python app removes all the photo metadata. How did you work around that?

[minch-dev]

@addohm He probably didn't, but you can use exiftoolgui to copy metadata back in bulk from original images. Although this requires resaving which isn't loseless if it's jpeg, or you have to use tiff. And yes, it's a lot of unnecessary steps that could be done right in a pipeline. The best thing would be to have a node that processes the images before feature extraction and applies the already generated mask or (ideally) generates one itself.

By the way, have anyone here ever used PhotoScan? The process of applying masks is pretty straightforward there, you just load them in bulk as monochrome png images (or alpha channel), then proceed with feature extraction. And it improves the quality of the generated model drastically. This feature doesn't seem to be the hardest to implement, although this functionality is crucial for the workflow.

[jaggzh]

The main issue with this is that the edge features ARE detected, especially with Akaze feature detection. (It's not as much with sift variants; although this might depend on the type of imagery you're working with).

Now, with the variation in manually-created masks, the akaze features might more or less be considered noise, but if they're sufficient they might interfere with Matching steps. This problem will potentially be worse with automated segmenting, which might produce masks along similar locations/feature lines in datasets.

What would be useful would be a way of taking a mask and, using, say, a settable Erode, remove all features with coordinates outside of the [eroded] mask. This would keep accuracy high as well as decrease computation time with all the extraneous useless (or worse) features during Matching.

I'm not sure if this is possible with the current setup. Moving the erosion to an external pipeline is possible, for simplicity, but the masks must be [in this method] able to be applied to the feature removal.

[natowi]

https://github.com/alicevision/Meshroom/wiki/New-Features-in-Meshroom-2023.1#1-image-masking