How to download image files from Laion-5b?

rom1504 / img2dataset

Easily turn large sets of image urls to an image dataset. Can download, resize and package 100M urls in 20h on one machine.

MIT License

3.71k stars 338 forks source link

Closed laolongboy closed 1 month ago

laolongboy commented 1 month ago

I got list like ["LAION-5B/laion2B-en/00065/3129797007897.0.jpg", ...]

so I can parse parquet id and file name from each element. How can I download each image in the list without download all parquets?

rom1504 commented 1 month ago

what parquet files do you mean?