Inputs must be sparse tensors when processing raw eval data

[Jaykob]

Hi! Thanks for sharing your very promising work!

I'm trying to follow your instructions to download and process (feature extraction) your evaluation data. However when calling extract_data.py as in your example, I'm getting the following output:

2020-07-13 15:09:05 824982bf9848 root[13462] INFO Starting feature extraction
2020-07-13 15:09:05 824982bf9848 root[13462] INFO ['./data/eval_data/3d_match/raw_data/kitchen', './data/eval_data/3d_match/raw_data/sun3d-home_at-home_at_scan1_2013_jan_1', './data/eval_data/3d_match/raw_data/sun3d-home_md-home_md_scan9_2012_sep_30', './data/eval_data/3d_match/raw_data/sun3d-hotel_uc-scan3', './data/eval_data/3d_match/raw_data/sun3d-hotel_umd-maryland_hotel1', './data/eval_data/3d_match/raw_data/sun3d-hotel_umd-maryland_hotel3', './data/eval_data/3d_match/raw_data/sun3d-mit_76_studyroom-76-1studyroom2', './data/eval_data/3d_match/raw_data/sun3d-mit_lab_hj-lab_hj_tea_nov_2_2012_scan1_erika']
2020-07-13 15:09:05 824982bf9848 root[13462] INFO 0 / 60: kitchen_000
/content/drive/My Drive/Dev/DL/colab_data/3D_multiview_reg/scripts/utils.py:118: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  coords = torch.tensor(coords, dtype=torch.int32)

Traceback (most recent call last):
  File "./scripts/extract_data.py", line 363, in <module>
    logger.info('Feature extraction completed')
  File "./scripts/extract_data.py", line 88, in extract_features_batch

  File "/content/drive/My Drive/Dev/DL/colab_data/3D_multiview_reg/scripts/utils.py", line 125, in extract_features
    return return_coords, model(stensor).F
  File "/usr/local/envs/lmpr/lib/python3.6/site-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File "/content/drive/My Drive/Dev/DL/colab_data/3D_multiview_reg/lib/descriptor/fcgf.py", line 257, in forward
    out = ME.cat((out_s4_tr, out_s4))
  File "/usr/local/envs/lmpr/lib/python3.6/site-packages/MinkowskiEngine-0.4.3-py3.6-linux-x86_64.egg/MinkowskiEngine/MinkowskiOps.py", line 67, in cat
    assert isinstance(s, SparseTensor), "Inputs must be sparse tensors."
AssertionError: Inputs must be sparse tensors.

Do you have an idea why I'm seeing this or am I missing something?

[zgojcic]

Hi, thanks for opening this issue, indeed it seems that there was a small bug in the fcgf code (to many brackets, probably a version issue). This should be fixed in commit 5a29912e37d8840376284269b751fd7f53efa347.

Could you please try it out and let me know if it fixes your problem.

Zan

[Jaykob]

Hi Zan! Yes, now it works :-) Thank you for the quick fix!

[Jaykob]

Sorry, I think I found some other minor bugs. Please let me know if I should rather open new issue(s)?

• lib/filtering/__init__.py has a misplaced comma in the imports
• lib/config.py tries to import and use non-existing multiview.py
• pairwise registration example call tries to load data from a non-existing path. I guess one 3d_match folder too much in the call? When I remove it I get the following error, though:

  File "./scripts/benchmark_pairwise_registration.py", line 417, in <module>
  eval_data, scene_info = make_pairwise_eval_data_loader(args)
  File "/content/drive/My Drive/Dev/DL/colab_data/3D_multiview_reg/scripts/utils.py", line 160, in make_pairwise_eval_data_loader
  dset = PrecomputedPairwiseEvalDataset(args)
  File "/content/drive/My Drive/Dev/DL/colab_data/3D_multiview_reg/lib/data.py", line 188, in __init__
  gt_pairs, gt_traj = read_trajectory(os.path.join(self.root,'raw_data', curr_scene_name, "gt.log"))
  File "/content/drive/My Drive/Dev/DL/colab_data/3D_multiview_reg/lib/utils.py", line 452, in read_trajectory
  with open(filename) as f:
  FileNotFoundError: [Errno 2] No such file or directory: './data/eval_data/3d_match/raw_data/.ipynb_checkpoints/gt.log'

[zgojcic]

Thanks Jaykob, I am sorry for the small bugs, guess this is what happens when one publishes only part of the code :)

You are completely right on all three points. I have changed __init__.py and config.py accordingly and updated Readme.md such that the call only contains one 3d_match.

The error that you are getting after changing all is probably because you tried running the jupyter notebook in the raw_data folder which created the .ipynb_checkpoints folder. Out code scans the raw_data folder and assumes that each folder in it represent a different scene, which has to contain the gt.log file in order to make the comparison.

Short answer: removing the .ipynb_checkpoints folder should do the trick.

If you find any other problems please feel free to either report them here or open a new issue :) Thank you again.

[Jaykob]

Thanks for the quick fixes, it works now :-)

Just a quick question: Do you think that your method (and the provided model) would be suited to register scans of a human foot? Or would you assume that its features (rather smooth, no edges, etc.) are too different from the training sets?

[zgojcic]

Hi Jaykob,

this is a bit hard to answer without actually seeing the point clouds. If they true to the metric scale, and the "size" of the resolution are similar (at least not much lower) than our data, I think that it should generalize to some extent. The only problem that I can see is the FCGF (voxel size that we use is 2.5 cm that could be important), if the feature descriptor works reasonably then it should be fine.

[zgojcic]

Closed due to inactivity. If you have some other questions please just open a new issue.