search

hustvl / VAD

[ICCV 2023] VAD: Vectorized Scene Representation for Efficient Autonomous Driving
https://arxiv.org/abs/2303.12077
Apache License 2.0
546 stars 50 forks source link

Not enough data sets? #23

Open h-enomoto opened 10 months ago

h-enomoto commented 10 months ago

Hello,

I hope this finds you well. I've been facing some challenges in successfully verifying the VAD, and I kindly seek your advice. I've followed the installation steps outlined in the documentation.

I have downloaded and extracted the necessary files from the NVIDIA website. 

can_bus.zip
nuScenes-map-expansion-v1.3.zip
v1.0-trainval01_blobs.tar
v1.0-trainval02_blobs.tar
v1.0-trainval03_blobs.tgz
v1.0-trainval04_blobs.tgz
v1.0-trainval05_blobs.tgz
v1.0-trainval06_blobs.tgz
v1.0-trainval07_blobs.tgz
v1.0-trainval08_blobs.tgz
v1.0-trainval09_blobs.tgz
v1.0-trainval10_blobs.tgz
v1.0-test_blobs.tar
v1.0-test_meta.tar
v1.0-trainval_meta.tgz

I executed the command provided under the "Prepare nuScenes data" section. python tools/data_converter/vad_nuscenes_converter.py nuscenes --root-path ./data/nuscenes --out-dir ./data/nuscenes --extra-tag vad_nuscenes --version v1.0 --canbus ./data

Unfortunately, this resulted in an error.

  File "tools/data_converter/vad_nuscenes_converter.py", line 988, in <module>
    nuscenes_data_prep(
  File "tools/data_converter/vad_nuscenes_converter.py", line 937, in nuscenes_data_prep
    create_nuscenes_infos(
  File "tools/data_converter/vad_nuscenes_converter.py", line 106, in create_nuscenes_infos
    train_nusc_infos, val_nusc_infos = _fill_trainval_infos(
  File "tools/data_converter/vad_nuscenes_converter.py", line 246, in _fill_trainval_infos
    mmcv.check_file_exist(lidar_path)
  File "/home/user1/miniconda3/envs/vad/lib/python3.8/site-packages/mmcv/utils/path.py", line 23, in check_file_exist
    raise FileNotFoundError(msg_tmpl.format(filename))
FileNotFoundError: file "./data/nuscenes/samples/LIDAR_TOP/n008-2018-08-01-16-03-27-0400__LIDAR_TOP__1533153857947444.pcd.bin" does not exist

While the directory does contain files, it appears that none of them are causing the error in question.

$ ls ~/VAD/data/nuscenes/samples/LIDAR_TOP/ -1 | wc -l
34149

Is it possible that a required file might be missing? I would greatly appreciate any guidance or support you can offer. Thank you for your time and consideration.

Best regards, enomoto

rb93dett commented 10 months ago

It seems that some data is missing in your nuscenes/samples/LIDAR_TOP directory, I run the same cmd and output like this:

$ ls data/nuscenes/samples/LIDAR_TOP/ -1 | wc -l
40157

You can try to re-download the nuScenes data or use our provided pre-processing data by following this README.

h-enomoto commented 10 months ago

Thank you very much, @rb93dett . I decompressed v1.0-test_blobs.tar and tried it, and the aforementioned error no longer occurred. I appreciate your advice!

However, I encountered a different issue as I proceeded. I executed the following command. (Since VAD_base.py does not exist, I used VAD_tiny_e2e.py)

CUDA_VISIBLE_DEVICES=0 python tools/test.py projects/configs/VAD/VAD_tiny_e2e.py /home/user1/VAD/ckpts/resnet50-19c8e357.pth --launcher none --eval bbox --tmpdir tmp

The result yielded the following error:

File "tools/test.py", line 294, in <module>
  main()
File "tools/test.py", line 274, in main
  print(dataset.evaluate(outputs['bbox_results'], **eval_kwargs))
File "/home/user1/VAD/projects/mmdet3d_plugin/datasets/nuscenes_vad_dataset.py", line 1786, in evaluate
  result_dict['ADE_'+cls] = all_metric_dict['ADE_'+cls] / all_metric_dict['cnt_ade_'+cls]
ZeroDivisionError: float division by zero

I tried VAD_tiny_stage_1.py, VAD_tiny_stage_2.py, VAD_base_e2e.py, VAD_base_stage_1.py, VAD_base_stage_2.py, but the result was the same.

I would be grateful for any help you could provide.

rb93dett commented 9 months ago

You only load a res50 pretrained checkpoint in your cmd, you should load our provided pretrained model checkpoint and use the corresponding config file (VAD-base or VAD-tiny), staged config and e2e config have the same model arch, only different training pipelines, so either one is ok, also remind to adjust the img_norm_cfg in the config file, detailed instructions please follow this README.

h-enomoto commented 9 months ago

I have read the README. Where is the config file(VAD-base or VAD-tiny)? It does not seem to exist in "projects/configs/VAD".

image

rb93dett commented 9 months ago

You only load a res50 pretrained checkpoint in your cmd, you should load our provided pretrained model checkpoint and use the corresponding config file (VAD-base or VAD-tiny), staged config and e2e config have the same model arch, only different training pipelines, so either one is ok, also remind to adjust the img_norm_cfg in the config file, detailed instructions please follow this README.

VAD-base or VAD-tiny indicates different model sizes here, I want to say that you should choose the right config (with different model sizes) corresponding to the pre-trained model you use, not mean the config names VAD-base or VAD-tiny.

h-enomoto commented 9 months ago

Thank you for your assistance. I have followed the steps outlined in the "Prepare nuScenes data" section of prepare_dataset.md. I have downloaded train and val and intend to use them as pre-trained models. Also, train's "Train VAD with 8 GPUs" is not running because it does not have the necessary resources. Could you kindly advise which config file I should use in the following command? 

CUDA_VISIBLE_DEVICES=0 python tools/test.py projects/configs/VAD/?????.py /home/user1/VAD/ckpts/resnet50-19c8e357.pth --launcher none --eval bbox --tmpdir tmp

Your guidance would be greatly appreciated.