Weird Mesh from provided Colab example

[antoinekeller]

After running the colab provided example, it came out a weird result (see attached screenshot from cloudcompare). I didnt get any error apart from

# run inference
!python inference.py --model results/release/semseg/final.ckpt --scenes data/sample/sample1/info.json --voxel_dim 208 208 80

0 1 sample sample1 0 250
/usr/local/lib/python3.6/dist-packages/detectron2/modeling/backbone/fpn.py:133: UserWarning: Mixed memory format inputs detected while calling the operator. The operator will output channels_last tensor even if some of the inputs are not in channels_last format. (Triggered internally at  /pytorch/aten/src/ATen/native/TensorIterator.cpp:924.)
  prev_features = lateral_features + top_down_features
0 1 sample sample1 25 250
0 1 sample sample1 50 250
0 1 sample sample1 75 250
0 1 sample sample1 100 250
0 1 sample sample1 125 250
0 1 sample sample1 150 250
0 1 sample sample1 175 250
0 1 sample sample1 200 250
0 1 sample sample1 225 250

Is this warning the root of the problem? How can I solve it?

Can anyone show the expected result from this example? Thx a lot

[zmurez-ml]

That looks like the expected result. The problem is there is no shading in your visualization which makes it hard to see the geometry (also make sure to view using backface culling since Atlas often produces closed meshes). Screenshoot below with flat shading and the result from COLMAP for comparison.

Atlas:

COLMAP:

[antoinekeller]

Thank you so much for your answer. Effectively, I had to able the shading option. But, I wonder now, why are my RGB colors so strange? In the video, the floor is kind of orange/red (carpet), but it appears green in my visualization??

[zmurez-ml]

The colors represent the semantic labels (using the NYU40 colormap). So green is floor, pink is table, yellow is chair, etc.

The code also supports a model the regresses rgb color but we do not provide a pretrained model with this head.https://github.com/magicleap/Atlas/blob/7fe755cd2ad31348a796cf215778f58e0c835821/atlas/heads3d.py#L211

[chetana02]

Hey, can you please share the .ply file that you got after running inference.py. Despite doing the same execution, I'm not able to open that file, as there are no vertices generated in the .ply file.

[HaFred]

Same here. And I am also wondering why is it done with --voxel_dim 208 208 80 in the readme for inference. Whereas, in the ~/atlas/config.py, the voxel configs are as below:

_C.VOXEL_DIM_TRAIN = [160,160,64]
_C.VOXEL_DIM_VAL = [256,256,96]
_C.VOXEL_DIM_TEST = [416,416,128]

Which one is correct for inference? Did you solve it? @antoinekeller Thank you very much.

[HaFred]

Hi @zmurez-ml , sorry I am a bit confused. May I know is the NYU40 dataset actually the NYUv2 in ScanNet dataset? According to the NYUv2 paper, it seems like the table is not pink but orange?

[dyy0205]

hi, can i ask which software did you use to show .ply file? thanks

[HaFred]

hi, can i ask which software did you use to show .ply file? thanks

Meshlab