search

CollaborativePerception / V2Xverse

77 stars 10 forks source link

Question About Generation Time and Storage Space of the Dataset #20

Open OctoberKat opened 3 months ago

OctoberKat commented 3 months ago

Hello! Thank you for your amazing work and open-source contributions. Generating the large training dataset is taking me a considerable amount of time (without parallel processing) and storage space. Could you please let me know how long it might take to complete and how much storage space it would require? Also, would it be possible to download the dataset? Thank you!

gjliu9 commented 2 months ago

Hello, thank you for your attention! The dataset indeed takes time to generate, it took us about 30h-40h to collect data across 108 routes (about 700G) on two 3090 GPUs with parallel processing. You may not need to collect data on all routes provided in our repository, a subset is helpful enough. Here are some tips that may be useful:

  1. Try parallel processing
  2. Reduce the number of agents
  3. Try collecting on a relatively small subset Our dataset is too large, so we will consider uploading a compact version.