Evaluation with mesh reconstruction obtained using different camera params

[aditya-vora]

Hello, I am trying to do an evaluation on DTU objects for my project. However, for some reasons I am not using the default camera params given by the DTU dataset, but I am reconstructing the camera params of DTU dataset (again) using SFM (from colmap) and then using those parameters to do a mesh reconstruction. Hence my reconstructed mesh is in a different coordinate system than expected, although I am getting correct reconstruction. Because of this I believe the script is filtering a lot of vertices in my mesh and hence I am getting the following error.

Traceback (most recent call last):
  File "./eval/eval.py", line 303, in <module>
    dist_d2s, idx_d2s = nn_engine.kneighbors(data_in_obs, n_neighbors=1, return_distance=True)
  File "/localhome/ava40/miniconda3/envs/neus/lib/python3.8/site-packages/sklearn/neighbors/_base.py", line 806, in kneighbors
    X = self._validate_data(X, accept_sparse="csr", reset=False, order="C")
  File "/localhome/ava40/miniconda3/envs/neus/lib/python3.8/site-packages/sklearn/base.py", line 546, in _validate_data
    X = check_array(X, input_name="X", **check_params)
  File "/localhome/ava40/miniconda3/envs/neus/lib/python3.8/site-packages/sklearn/utils/validation.py", line 931, in check_array
    raise ValueError(
ValueError: Found array with 0 sample(s) (shape=(0, 3)) while a minimum of 1 is required by NearestNeighbors.
compute data2stl:  67%|████████████████████████████████████████████                      | 6/9 [00:16<00:08,  2.81s/it]

Because of this I am not able to do the evaluation. Is there a way I can handle this case in the script?

Thanks.

[jzhangbs]

Hi, the script assumes the two meshes are aligned. You can do registration on two set of cameras or meshes to align them first.

[aditya-vora]

Thanks for the response. Yes I figured out that I need to do camera registration. However, on doing camera registration when I apply mesh transformation, it seems my mesh is not getting exactly aligned with the ground truth. There seems some translation error because of which my CD distance numbers are poor even though my mesh reconstruction results are good. Is there a way I can handle this? Thanks.

[jzhangbs]

It's very likely that the two results are not aligned perfectly. Basically the evaluation script assume you use the ground truth cameras. I don't know your use case, but try to use ground truth camera if possible

[aditya-vora]

Yea I thought the same. Its the registration error. I am trying to use GT cams now. Thanks for your response.