search

opendilab / LMDrive

[CVPR 2024] LMDrive: Closed-Loop End-to-End Driving with Large Language Models
Apache License 2.0
629 stars 51 forks source link

Dataset download problem #6

Closed ltp1995 closed 7 months ago

ltp1995 commented 9 months ago

Hi, due to policy restrictions in China, it is no longer possible to directly download your dataset through Hugging Face ( https://huggingface.co/datasets/OpenDILabCommunity/LMDrive/tree/main). May I ask if there is a Baidu Cloud link (or some other source links) available?

deepcs233 commented 8 months ago

Hi! You can try this website(opendatalab): https://opendatalab.org.cn/deepcs233/LMDrive/tree/main

We are uploading the dataset to this platform. Until now, we have uploaded about 40% dataset.

ltp1995 commented 8 months ago

Hi! You can try this website(opendatalab): https://opendatalab.org.cn/deepcs233/LMDrive/tree/main

We are uploading the dataset to this platform. Until now, we have uploaded about 40% dataset.

Got it, thanks for your kind reply!

weimengchuan commented 8 months ago

Hi! You can try this website(opendatalab): https://opendatalab.org.cn/deepcs233/LMDrive/tree/main

We are uploading the dataset to this platform. Until now, we have uploaded about 40% dataset.

Hi! Is the data upload completed currently?

deepcs233 commented 8 months ago

Hi! We have uploaded about 80% dataset. It should be finished in this week!

weimengchuan commented 8 months ago

Hi! We have uploaded about 80% dataset. It should be finished in this week!

Thanks a lot!

weimengchuan commented 8 months ago

Hi! I have another 2 questions.

  1. Is it best to wait until your data upload is completed before starting to download it? If we start downloading now, will it result in missing some data?
  2. How can we use the CLI or SDK to download a part of the whole dataset? For example, how to download only LMDrive/data/Town01? Thanks in advance!
deepcs233 commented 8 months ago

Hi! If you haven't started to download the dataset. I recommend you wait until the uploading is ready.

About the question 2, I am not familiar with the openxlab platform. But it looks ok, and you can refer to https://openxlab.org.cn/docs/datasets/%E4%B8%8B%E8%BD%BD%E6%95%B0%E6%8D%AE%E9%9B%86.html

weimengchuan commented 8 months ago

Hi! If you haven't started to download the dataset. I recommend you wait until the uploading is ready.

About the question 2, I am not familiar with the openxlab platform. But it looks ok, and you can refer to https://openxlab.org.cn/docs/datasets/%E4%B8%8B%E8%BD%BD%E6%95%B0%E6%8D%AE%E9%9B%86.html

Got it, thanks!

weimengchuan commented 8 months ago

Hi! We have uploaded about 80% dataset. It should be finished in this week!

Hi! Have you finished the dataset uploading?

deepcs233 commented 8 months ago

Hi! I'm very sorry for the delay. We met some network problems last week. So we need more 2-3 days to upload them. Maybe you can download part of them and debug your training pipeline:)

deepcs233 commented 8 months ago

@ltp1995 @weimengchuan Hi! We have uploaded the whole dataset to openxlab now!

ForestWang commented 8 months ago

@ltp1995 @weimengchuan Hi! We have uploaded the whole dataset to openxlab now!

could you upload the md5sum for each tar file? so many files, maybe some files are not full downloaded. thanks.

deepcs233 commented 8 months ago

Hi! We have checked the uploaded dataset. And if several tar files don't work, it will not affect the finial training results.