Scannet Problem

[github-yizhang]

Thanks for your wonderful work on keypoint. I am trying to follow your work, but some issues about scannet bother me. 1. I think there is a spelling mistake in ./doc/usage/dataset.md, when download Scannet dataset, update the path in ./etc/datasets/image-net/{training-all,test}.yaml. should be corrected. 2. After I set up my python environment, I test my environment using "./bin/run_tests", but some mistake are reported as

ERROR: test_scannet_loftr (datasets.scannet.loftr_test._UnitTests)
Traceback (most recent call last):
  File "/home/zhangy/Desktop/silk/lib/datasets/scannet/loftr_test.py", line 27, in test_scannet_loftr
    ds = _UnitTests._get_dataset()
  File "/home/zhangy/Desktop/silk/lib/datasets/scannet/loftr_test.py", line 24, in _get_dataset
    return scannet_all(DATASET_PATH, TRAIN_NPZ_PATH, MODE, MIN_OVERLAP_SCORE_TRAIN)
  File "/home/zhangy/Desktop/silk/silk/datasets/scannet/loftr.py", line 195, in scannet_all
    return ConcatDataset(
  File "/home/zhangy/.conda/envs/silk/lib/python3.8/site-packages/torch/utils/data/dataset.py", line 397, in __init__
    self.datasets = list(datasets)
  File "/home/zhangy/Desktop/silk/silk/datasets/scannet/loftr.py", line 196, in <genexpr>
    ScanNetDataset(
  File "/home/zhangy/Desktop/silk/silk/datasets/scannet/loftr.py", line 242, in __init__
    self.data_names = self._h5file["name"]
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "/home/zhangy/.conda/envs/silk/lib/python3.8/site-packages/h5py/_hl/group.py", line 305, in __getitem__
    oid = h5o.open(self.id, self._e(name), lapl=self._lapl)
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "h5py/h5o.pyx", line 190, in h5py.h5o.open
KeyError: "Unable to open object (object 'name' doesn't exist)"

I am wondering do I need to do some pre-processing on scannet first，I download scannetv2 and use its raw data.

3. I am trying to train SiLK on Scannet dataset, some mistake still happen, As follows

RuntimeError: Caught RuntimeError in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "/home/zhangy/.conda/envs/silk/lib/python3.8/site-packages/torch/utils/data/_utils/worker.py", line 287, in _worker_loop
    data = fetcher.fetch(index)
  File "/home/zhangy/.conda/envs/silk/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 52, in fetch
    return self.collate_fn(data)
  File "/home/zhangy/.conda/envs/silk/lib/python3.8/site-packages/pytorch_lightning/utilities/auto_restart.py", line 474, in _capture_metadata_collate
    data = default_collate(samples)
  File "/home/zhangy/Desktop/silk/silk/transforms/tensor.py", line 150, in __call__
    transformed = [self._transform(it) for it in item]
  File "/home/zhangy/Desktop/silk/silk/transforms/tensor.py", line 150, in <listcomp>
    transformed = [self._transform(it) for it in item]
  File "/home/zhangy/Desktop/silk/silk/transforms/abstract.py", line 467, in __call__
    item = transform(item)
  File "/home/zhangy/Desktop/silk/silk/transforms/abstract.py", line 423, in __call__
    ensure_is_instance(item, tuple)
  File "/home/zhangy/Desktop/silk/silk/config/core.py", line 247, in ensure_is_instance
    raise RuntimeError(
RuntimeError: instance should be of type tuple, not silk.datasets.scannet.helper.Frame

In conclusion, it seems the way I use scannet dataset is wrong. It would be extremely grateful if you can provide me the detail between downloading scannet and training on scannet. Thank you again for your devotion to this outstanding work, and I look forward to your early reply.

[gleize]

Hi @github-yizhang,

1. I think there is a spelling mistake in ./doc/usage/dataset.md, when download Scannet dataset, update the path in ./etc/datasets/image-net/{training-all,test}.yaml. should be corrected.

Thanks a lot for pointing that out. It will be fixed soon (c.f. https://github.com/facebookresearch/silk/pull/46/commits/bb37262615e804f32b17a4a8b84d7072b41edc28).

2. After I set up my python environment, I test my environment using "./bin/run_tests", but some mistake are reported as [...]

That unittest tests the LoFTR's version of the ScanNet dataset class. It will only works on the FAIR internal infrastructure (because of the hardcoded paths). Since we don't use that version to train or test SiLK, you can simply comment or remove that test file (I might remove it in a following commit as well). Was that the only unit test that failed ?

3. I am trying to train SiLK on Scannet dataset, some mistake still happen, As follows

For ScanNet only, there are two places (c.f. here and below) that should be uncommented to make it work. Thanks for pointing that out, I'll add it to the documentation as well.

[github-yizhang]

Thank you for your reply, it solved all the issues I was facing.

2. It was the only failed test. After I uncommment the loftr_test, all of the unit test passed successfully. Thank you very much!