Question about multi_frame training in Waymo

[YoushaaMurhij]

Question: How to train 3D model using multi_frame config on Waymo dataset? I prepared the data for multi_frame training using :

python -m pcdet.datasets.waymo.waymo_dataset --func create_waymo_infos \
    --cfg_file tools/cfgs/dataset_configs/waymo_dataset_multiframe.yaml

But, I am not sure if only setting _BASECONFIG: cfgs/dataset_configs/waymo_dataset_multiframe.yaml is enough to do the job! And how to select how many frames ot be used?

Thanks!

[chenshi3]

you can adjust the selected frames according this key: TIME_RANGE: [0.3, 0.0] # 0.3s-0.0s indicates 4 frames

[YoushaaMurhij]

@chenshi3, @Haiyang-W , I tried to train the 3D model with multiframe setup (by setting '_BASECONFIG: cfgs/dataset_configs/waymo_dataset_multiframe.yaml'). But, this error showed up:

2024-03-21 08:59:22,993   INFO  **********************Start training cfgs/dsvt_models/dsvt_3D_D512e_multiframe(default)**********************
Traceback (most recent call last):                                                                                                                                                                                                                            
  File "/home/user/dsvt/tools/train.py", line 270, in <module>                                                                                                                                                                      | 0/15809 [00:00<?, ?it/s]
    main()
  File "/home/user/dsvt/tools/train.py", line 214, in main
    train_model(
  File "/home/user/dsvt/tools/train_utils/train_utils.py", line 236, in train_model
    accumulated_iter = train_one_epoch(
  File "/home/user/dsvt/tools/train_utils/train_utils.py", line 51, in train_one_epoch
    batch = next(dataloader_iter)
  File "/usr/local/lib/python3.10/dist-packages/torch/utils/data/dataloader.py", line 634, in __next__
    data = self._next_data()
  File "/usr/local/lib/python3.10/dist-packages/torch/utils/data/dataloader.py", line 1346, in _next_data
    return self._process_data(data)
  File "/usr/local/lib/python3.10/dist-packages/torch/utils/data/dataloader.py", line 1372, in _process_data
    data.reraise()
  File "/usr/local/lib/python3.10/dist-packages/torch/_utils.py", line 644, in reraise
    raise exception
AssertionError: Caught AssertionError in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/torch/utils/data/_utils/worker.py", line 308, in _worker_loop
    data = fetcher.fetch(index)
  File "/usr/local/lib/python3.10/dist-packages/torch/utils/data/_utils/fetch.py", line 51, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/usr/local/lib/python3.10/dist-packages/torch/utils/data/_utils/fetch.py", line 51, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/user/dsvt/tools/../pcdet/datasets/waymo/waymo_dataset.py", line 400, in __getitem__
    data_dict = self.prepare_data(data_dict=input_dict)
  File "/home/user/dsvt/tools/../pcdet/datasets/dataset.py", line 160, in prepare_data
    data_dict = self.data_augmentor.forward(
  File "/home/user/dsvt/tools/../pcdet/datasets/augmentor/data_augmentor.py", line 266, in forward
    data_dict = cur_augmentor(data_dict=data_dict)
  File "/home/user/dsvt/tools/../pcdet/datasets/augmentor/database_sampler.py", line 501, in __call__
    data_dict = self.add_sampled_boxes_to_scene(
  File "/home/user/dsvt/tools/../pcdet/datasets/augmentor/database_sampler.py", line 426, in add_sampled_boxes_to_scene
    assert obj_points.shape[-1] == points.shape[-1] + 1
AssertionError

Do you have any suggestion to fix it?

Thanks

[chenshi3]

keep FILTER_OBJ_POINTS_BY_TIMESTAMP = True

[YoushaaMurhij]

@chenshi3 , It is already True. I am actuallly using your default config files