IndexError during inference

Checklist

[X] I have searched for similar issues.
[X] I have tested with the latest development wheel.
[X] I have checked the release documentation and the latest documentation (for main branch).

Describe the issue

I have trained RandlaNet architecture on SensatUrban dataset by creating a dedicated dataset class using toronto3d template. I have trained the architecture with a grid size of 0.05 but when I try to perform an inference on the test set I get IndexError in the update_tests function. I think it might be related to the grid size but I can not understand why.

Steps to reproduce the bug

Config file

dataset:
  name: SensatUrban
  cache_dir: ./dataset_cache
  dataset_path: data/digitwin/sensatUrban/ply
  class_weights: [438547425, 543847609, 860487257, 20606217, 3044807, 49452238, 498977, 131470199, 26669467, 37557130, 43503086, 174125, 7088841]
  num_classes: 13
  num_points: 65536
  train_files:
  - train/birmingham_block_10.ply
  - train/birmingham_block_11.ply
  - train/birmingham_block_12.ply
  - train/birmingham_block_13.ply
  - train/birmingham_block_3.ply
  - train/birmingham_block_4.ply
  - train/birmingham_block_6.ply
  - train/cambridge_block_12.ply
  - train/cambridge_block_13.ply
  - train/cambridge_block_14.ply
  - train/cambridge_block_17.ply
  - train/cambridge_block_18.ply
  - train/cambridge_block_19.ply
  - train/cambridge_block_2.ply
  - train/cambridge_block_20.ply
  - train/cambridge_block_21.ply
  - train/cambridge_block_23.ply
  - train/cambridge_block_25.ply
  - train/cambridge_block_26.ply
  - train/cambridge_block_28.ply
  - train/cambridge_block_3.ply
  - train/cambridge_block_32.ply
  - train/cambridge_block_33.ply
  - train/cambridge_block_34.ply
  - train/cambridge_block_4.ply
  - train/cambridge_block_6.ply
  - train/birmingham_block_0.ply
  - train/birmingham_block_7.ply
  - train/birmingham_block_9.ply
  - train/cambridge_block_0.ply
  - train/cambridge_block_1.ply
  - train/cambridge_block_8.ply
  - train/cambridge_block_9.ply
  val_files:
  - train/birmingham_block_1.ply
  - train/birmingham_block_5.ply
  - train/cambridge_block_10.ply
  - train/cambridge_block_7.ply
  test_files:
  - test/birmingham_block_2.ply
  - test/birmingham_block_8.ply
  - test/cambridge_block_15.ply
  - test/cambridge_block_22.ply
  - test/cambridge_block_16.ply
  - test/cambridge_block_27.ply
  test_result_folder: ./test
  use_cache: true
  steps_per_epoch_train: 100
  steps_per_epoch_valid: 10
  use_cache: true
model:
  name: RandLANet
  batcher: DefaultBatcher
  ckpt_path: # path/to/your/checkpoint
  num_neighbors: 16
  num_layers: 5
  num_points: 65536
  num_classes: 13
  ignored_label_inds: []
  sub_sampling_ratio: [4, 4, 4, 4, 2]
  in_channels: 6
  dim_input: 6
  dim_features: 8
  dim_output: [16, 64, 128, 256, 512]
  grid_size: 0.05
  augment:
    recenter:
      dim: [0, 1, 2]
    normalize:
      points:
        method: linear
pipeline:
  name: SemanticSegmentation
  optimizer:
    lr: 0.01
  batch_size: 4
  main_log_dir: experiments/sensat
  max_epoch: 800
  save_ckpt_freq: 1
  scheduler_gamma: 0.99
  test_batch_size: 1
  num_workers: 0
  pin_memory: false
  val_batch_size: 2
  experiment: randlanet_sensat_weight_paper
  summary:
    record_for: []
    max_pts:
    use_reference: false
    max_outputs: 1

python Open3D-ML/scripts/run_pipeline.py torch \ --split test \ -c {CFG_PATH} \ --model.ckpt_path {MODEL_PATH} \ --dataset.test_result_folder {OUT_PATH}


### Error message

```python
Traceback (most recent call last):
  File "Open3D-ML/scripts/run_pipeline.py", line 301, in <module>
    sys.exit(main())
  File "Open3D-ML/scripts/run_pipeline.py", line 213, in main
    pipeline.run_test()
  File "/nfs/home/blanco/digitwin/Open3D-ML/ml3d/torch/pipelines/semantic_segmentation.py", line 242, in run_test
    self.update_tests(test_sampler, inputs, results)
  File "/nfs/home/blanco/digitwin/Open3D-ML/ml3d/torch/pipelines/semantic_segmentation.py", line 323, in update_tests
    test_labels = np.argmax(self.test_probs[self.curr_cloud_id][proj_inds], 1)
IndexError: index 10379412 is out of bounds for axis 0 with size 3444876

Expected behavior

Since the training completes smoothly I would expect the inference will go smoothly as well.

Open3D, Python and System information

- Operating system: Ubuntu 22.04.4 LTS
- Python version: Python 3.8.5
- Open3D version: 0.16.0
- System type: x86_64
- Is this remote workstation?: yes
- How did you install Open3D?: pip

Additional information

No response

isl-org / Open3D-ML