search

chenweikai / 3PSDF

66 stars 6 forks source link

marching cube with mask? #1

Closed StevenLiuWen closed 2 years ago

StevenLiuWen commented 2 years ago

Hi, @chenweikai,

Thanks for sharing this simple and efficient work for handling open-surface implicit neural representation via the NULL space.

The surface extraction step in the paper and implementation is firstly using the normal marching cube and then filtering the nan vertices and faces.

I am curious about that Could we use the marching cube with a (NULL-space) mask? Since the recent scikit-image supports https://scikit-image.org/docs/dev/api/skimage.measure.html?highlight=marching_cubes#skimage.measure.marching_cubes with mask parameters, if I understand correctly, the mask is (predicts == 2).

Looking forward to your reply and thanks in advance.

chenweikai commented 2 years ago

Hi, @chenweikai,

Thanks for sharing this simple and efficient work for handling open-surface implicit neural representation via the NULL space.

The surface extraction step in the paper and implementation is firstly using the normal marching cube and then filtering the nan vertices and faces.

I am curious about that Could we use the marching cube with a (NULL-space) mask? Since the recent scikit-image supports https://scikit-image.org/docs/dev/api/skimage.measure.html?highlight=marching_cubes#skimage.measure.marching_cubes with mask parameters, if I understand correctly, the mask is (predicts == 2).

Looking forward to your reply and thanks in advance.

Hi @StevenLiuWen,

Thank you for your interest in our work!

After 3PSDF is computed, it is indeed can be converted to a form of marked Marching cubes to enable the convenient use of the off-the-shelf tools, e.g. the scikit-image toolbox that you mentioned. It is one of the main advantages of 3PSDF -- easy to use and fully compatible with existing field-to-mesh techniques.

However, I wish to emphasize that learning 3PSDF is not as easy as just learning a mask for SDF. We do mention that there is an alternative framework to learn 3PSDF in Section A of the supplemental materials. That is to train a binary classification together with regression. The binary classification generates the narrow-band mask for distinguishing Null v.s. Valid regions, while the regression branch regresses the exact location of the surface. While such a naive masked SDF method works well for overfitting, it would lead to a misalignment issue as shown in Figure 11 of our arxiv-version paper.

The main reason is that the classification and regression have different task natures, in which their decision boundaries have a slight offset from each other. However, in a relatively high resolution where the width of the mask is pretty narrow, such an offset would incur a misalignment between the decision boundaries of these two branches. This would lead to incomplete results as shown in Figure 11, despite that both the classification and regression have generated plausible shapes. However, the 3-way classification wouldn't have such an issue as we only need a single branch to do all the job.:-)

In practice, we found that 3-way classification is much easier to learn compared to regression. We have tried to apply NDF in single-view reconstruction -- NDF cannot converge or generate plausible results.

StevenLiuWen commented 2 years ago

Got it and really appreciate your further explanation about the learning strategy of 3PSDF.