search

MegaScenes / dataset

45 stars 0 forks source link

how to download the dataset #1

Closed xuxumiao777 closed 4 hours ago

xuxumiao777 commented 2 days ago

Nice work! but the readme is too tricky. When I use wget http:*** , I can just download a index.html. Could you please tell me a simple way to download your dataset? Thank you.

jianglh-WHU commented 2 days ago

I also wanna know how to download the dataset by base URL.

j0seo commented 17 hours ago

I successfully downloaded the dataset using the command aws s3 sync --no-sign-request s3://megascenes/ ./ after installing AWS CLI.

jot-jt commented 8 hours ago

Thanks for the feedback, we will update the documentation with download instructions. I recommend downloading the dataset using the AWS CLI command as j0seo mentioned.

wget can only download singular files, while the AWS CLI can recursively download all files in a directory.

To download a file with wget, an example is wget https://megascenes.s3.us-west-2.amazonaws.com/databases/descriptors/000/000/descriptors.db.gz. But multiple commands need to be run to download the entire descriptors folder.

jianglh-WHU commented 5 hours ago

I successfully downloaded the dataset using the command aws s3 sync --no-sign-request s3://megascenes/ ./ after installing AWS CLI.

It works!