Haiyang-W / DSVT

[CVPR2023] Official Implementation of "DSVT: Dynamic Sparse Voxel Transformer with Rotated Sets"
https://arxiv.org/abs/2301.06051
Apache License 2.0
353 stars 28 forks source link

IndexError in test using nuscenes #45

Closed cjfghddl0206 closed 1 year ago

cjfghddl0206 commented 1 year ago

After training was completed using 3090x3 and it took 9 to 10 days, the evaluation did not run and stopped in the middle at epoch 20. After the value came out as shown in the photo, after loading 6019 data in the data val, loading the groud truth and filtering the prediction, an index error appeared. Screenshot from 2023-07-07 17-54-47

Haiyang-W commented 1 year ago

Please try to run the inference code first based on our pretrained model weight. If the performance can match the scores reported in our paper, it means that there is no problem with the data processing.

Haiyang-W commented 1 year ago

After training was completed using 3090x3 and it took 9 to 10 days, the evaluation did not run and stopped in the middle at epoch 20. After the value came out as shown in the photo, after loading 6019 data in the data val, loading the groud truth and filtering the prediction, an index error appeared. Screenshot from 2023-07-07 17-54-47

BTW, your training is too slow, the whole training process should be around 1-2 days.

cjfghddl0206 commented 1 year ago

After training was completed using 3090x3 and it took 9 to 10 days, the evaluation did not run and stopped in the middle at epoch 20. After the value came out as shown in the photo, after loading 6019 data in the data val, loading the groud truth and filtering the prediction, an index error appeared. Screenshot from 2023-07-07 17-54-47

BTW, your training is too slow, the whole training process should be around 1-2 days.

Isn't it slow because I use three rtx3090 gpu instead of a100?

Haiyang-W commented 1 year ago

After training was completed using 3090x3 and it took 9 to 10 days, the evaluation did not run and stopped in the middle at epoch 20. After the value came out as shown in the photo, after loading 6019 data in the data val, loading the groud truth and filtering the prediction, an index error appeared. Screenshot from 2023-07-07 17-54-47

BTW, your training is too slow, the whole training process should be around 1-2 days.

Isn't it slow because I use three rtx3090 gpu instead of a100?

I mean rtx 3090 is only 1 day (8 gpus). For A100, only 14h.

Haiyang-W commented 1 year ago

This issue seems to have been solved and will be closed. If you have any questions, please do not hesitate to ask, we will reply as soon as possible.