Add image downloading script

allenai / mmc4

MultimodalC4 is a multimodal extension of c4 that interleaves millions of images with text.

MIT License

901 stars 34 forks source link

Closed VegB closed 1 year ago

VegB commented 1 year ago

Add a script to download images with multiprocessing for a given shard jsonl