nmoehrle / mvs-texturing

Algorithm to texture 3D reconstructions from multi-view stereo images
Other
931 stars 328 forks source link

Texture Masking #138

Open ryanlukejohns opened 4 years ago

ryanlukejohns commented 4 years ago

What would be the best approach for managing texture masking of known obstacles that appear in the vast majority of images? The paper makes it clear that you ``explicitly want to avoid'' marked occlusion regions, but is there a strategy that might be used in the case that you can easily automatically mask objects in advance? I.E. if a large occlusion is blocking the frame in the majority of views, but you do not want to include it in the texture, is it possible to pre-mask it in any way (i.e. setting those pixels to a fixed color or transarency? Or setting them to arbitrarily different colors so that they are less likely to influence the consensus?) Any advice here would be very much appreciated!

nmoehrle commented 4 years ago

At the moment there are some assumptions on the visibility masks (e.g. quasi convex) that would need reconsideration if one wants to introduce arbitrary masks. However, it should not be too difficult to hack something into the computation of the data cost indicating whether the face should be texturable by the given view - in fact you could simply set the quality to 0 if a pixel of a certain color is encountered. I am unsure if this is something sustainable but I encourage you to look at https://github.com/nmoehrle/mvs-texturing/blob/master/libs/tex/calculate_data_costs.cpp#L222 and https://github.com/nmoehrle/mvs-texturing/blob/master/libs/tex/texture_view.cpp#L135 this is where you could hack it in.