search

rom1504 / img2dataset

Easily turn large sets of image urls to an image dataset. Can download, resize and package 100M urls in 20h on one machine.
MIT License
3.74k stars 341 forks source link

Support read image from local path #428

Open thesby opened 6 months ago

thesby commented 6 months ago

My dataset is on disk, and I want to use img2dataset to convert it as webdataset. But img2dataset cannot read local path images.

rom1504 commented 6 months ago

You can turn up a local http server (nginx is a good bet) then have a CSV or parquet file with something like "http://localhost/myimg124.jpg" in it and then give that csv to img2dataset

On Wed, May 29, 2024, 9:46 AM Luo_yuansheng @.***> wrote:

My dataset is on disk, and I want to use img2dataset to convert it as webdataset. But img2dataset cannot read local path images.

— Reply to this email directly, view it on GitHub https://github.com/rom1504/img2dataset/issues/428, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAR437UL5APKYT6LRYYLGDDZEWBTXAVCNFSM6AAAAABIOLSEF2VHI2DSMVQWIX3LMV43ASLTON2WKOZSGMZDENRRGUYTQNQ . You are receiving this because you are subscribed to this thread.Message ID: @.***>

rom1504 commented 6 months ago

We could also have direct support for local files but honestly it would not be very different from the above solution

On Wed, May 29, 2024, 10:42 AM Romain Beaumont @.***> wrote:

You can turn up a local http server (nginx is a good bet) then have a CSV or parquet file with something like "http://localhost/myimg124.jpg" in it and then give that csv to img2dataset

On Wed, May 29, 2024, 9:46 AM Luo_yuansheng @.***> wrote:

My dataset is on disk, and I want to use img2dataset to convert it as webdataset. But img2dataset cannot read local path images.

— Reply to this email directly, view it on GitHub https://github.com/rom1504/img2dataset/issues/428, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAR437UL5APKYT6LRYYLGDDZEWBTXAVCNFSM6AAAAABIOLSEF2VHI2DSMVQWIX3LMV43ASLTON2WKOZSGMZDENRRGUYTQNQ . You are receiving this because you are subscribed to this thread.Message ID: @.***>