Error when testing with Scannet v2

[VuGiaHuy-0808]

Hi, thanks for your great works. Currently I want to to test the model with Scannet test dataset. I already preprocessing test dataset and everything still works. But when I run tools/test.py I got below error:

File "tools/test.py", line 151, in main() File "tools/test.py", line 146, in main runner.test() File "/opt/conda/lib/python3.7/site-packages/mmengine/runner/runner.py", line 1823, in test dict_keys(['lidar_points', 'instances', 'super_pts_path', 'num_pts_feats', 'lidar_path', 'seg_label_mapping', 'eval_ann_info', 'sample_idx', 'points']) metrics = self.test_loop.run() # type: ignore File "/opt/conda/lib/python3.7/site-packages/mmengine/runner/loops.py", line 434, in run for idx, data_batch in enumerate(self.dataloader): File "/opt/conda/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 521, in next data = self._next_data() File "/opt/conda/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1203, in _next_data dict_keys(['lidar_points', 'instances', 'super_pts_path', 'num_pts_feats', 'lidar_path', 'seg_label_mapping', 'eval_ann_info', 'sample_idx', 'points']) return self._process_data(data) File "/opt/conda/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1229, in _process_data data.reraise() File "/opt/conda/lib/python3.7/site-packages/torch/_utils.py", line 434, in reraise raise exception KeyError: Caught KeyError in DataLoader worker process 0. Original Traceback (most recent call last): File "/opt/conda/lib/python3.7/site-packages/torch/utils/data/_utils/worker.py", line 287, in _worker_loop data = fetcher.fetch(index) File "/opt/conda/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 49, in fetch data = [self.dataset[idx] for idx in possibly_batched_index] File "/opt/conda/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 49, in data = [self.dataset[idx] for idx in possibly_batched_index] File "/opt/conda/lib/python3.7/site-packages/mmengine/dataset/base_dataset.py", line 403, in getitem data = self.prepare_data(idx) File "/opt/conda/lib/python3.7/site-packages/mmdet3d/datasets/seg3d_dataset.py", line 305, in prepare_data return super().prepare_data(idx) File "/opt/conda/lib/python3.7/site-packages/mmengine/dataset/base_dataset.py", line 793, in prepare_data return self.pipeline(data_info) File "/opt/conda/lib/python3.7/site-packages/mmengine/dataset/base_dataset.py", line 60, in call data = t(data) File "/opt/conda/lib/python3.7/site-packages/mmcv/transforms/base.py", line 12, in call return self.transform(results) File "/home/mmdetect3d/oneformer3d/loading.py", line 65, in transform results = super().transform(results) File "/opt/conda/lib/python3.7/site-packages/mmdet3d/datasets/transforms/loading.py", line 1058, in transform results = self._load_masks_3d(results) File "/opt/conda/lib/python3.7/site-packages/mmdet3d/datasets/transforms/loading.py", line 919, in _load_masks_3d pts_instance_mask_path = results['pts_instance_mask_path'] KeyError: 'pts_instance_mask_path'

I checked the results variable in oneformer3d/loading.py and get its keys value:

dict_keys(['lidar_points', 'instances', 'super_pts_path', 'num_pts_feats', 'lidar_path', 'seg_label_mapping', 'eval_ann_info', 'sample_idx', 'points'])

I think my test data is missing something and that leads to failure in dataloader. I'm not familiar with mmdetection3d so pls give me some advices. Thank you so much.

[filaPro]

Hi @VuGiaHuy-0808, Have you tried testing our model on ScanNet train or validation split? The test split doesn't have annotations. To achieve metrics on it you need to submit the predictions to the online benchmark.

[VuGiaHuy-0808]

I'm testing the model on validation split. I checked and added annotations to test split but now I'm faced with a new issue related to attention module:

File "/home/mmdetect3d/oneformer3d/querydecoder.py", line 52, in forward output, = self.attn(queries[i], k, v, attn_mask=attn_mask) File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl return forward_call(*input, **kwargs) File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/activation.py", line 1010, in forward attn_mask=attn_mask) File "/opt/conda/lib/python3.7/site-packages/torch/nn/functional.py", line 4968, in multi_head_attention_forward tgt_len, bsz, embed_dim = query.shape ValueError: not enough values to unpack (expected 3, got 2)

The query shape is ([256, 957]) Currently, I just want to visualize prediction results as you reported in your paper :

[VuGiaHuy-0808]

Cause is different torch version, I use torch 1.10. I will close the issue. Merry christmas btw.