search

AILab-CVC / SEED-X

Multimodal Models in Real World
Other
396 stars 16 forks source link

Details about Data Preprocessing #13

Open BolinLai opened 4 months ago

BolinLai commented 4 months ago

Thank you for your awesome work and codes! I want to ask more about data preprocessing to train the model. Can you let me know

geyuying commented 3 months ago

Hi, we currently support the following dataloader with the specified data structure.

For "build_llava_jsonl_datapipes" dataloader, each folder stores a number of jsonl files, each jsonl file contains 10K pieces of content, with an example of the content as follows:

{"image": "coco/train2017/000000033471.jpg", "data": ["What are the colors of the bus in the image?", "The bus in the image is white and red.", "What feature can be seen on the back of the bus?", "The back of the bus features an advertisement.", "Is the bus driving down the street or pulled off to the side?", "The bus is driving down the street, which is crowded with people and other vehicles."]}

For "build_caption_datapipes_with_pixels" dataloder, each folder stores a number of .tar files and reads image-text pairs in the form of webdataset.

For "build_single_turn_edit_datapipes" dataloder, each folder stores a number of jsonl files, each jsonl file contains 10K pieces of content, with an example of the content as follows:


{"source_image": "source_images/f6f4d0669694df5b.jpg", "target_image": "target_images/f6f4d0669694df5b.jpg", "instruction": "Erase the car that is parked in front of the Roebuck building."}
arpitbansal297 commented 2 months ago

Hi! Thanks for this awesome work. Can you let me know where to download the data from? https://huggingface.co/datasets/AILab-CVC/SEED-Data-Edit is empty The instructions given in https://huggingface.co/datasets/AILab-CVC/SEED-Data-Edit-Part1-Openimages says "After downloading the data, you first need to reassemble the split files back into the original .tar.gz file as below, and then unzip the files." Can you please give more instructions on where to download this data from and how to process it to use the huggingface dataset.

Thanks in advance.