DatasetGenerationError: An error occurred while generating the dataset

[DoctorTar]

I tried to run train_instruct_pix2pix_sdxl.py with my local data that looks like this: online_data/ 001input.jpg, 001edited.jpg, etc with a metadata.jsonl file following instructions in here, and its lines look like : {"input_image": "001input.jpg", "prompts": "1 young girl, chibi, curly double braids, black hair, White blouse with brown tie, sailor collar, medium build, oval face, light skin, , holding a cup of tea, solo, upper body, white background, super high definition, 32k, ", "file_name": "001edited.jpg"}

then I run train_instruct_pix2pix_sdxl.py with :

accelerate launch --mixed_precision="fp16" --multi_gpu train_instruct_pix2pix_sdxl.py \ --pretrained_model_name_or_path='./sd_models/stabilityai__stable-diffusion-xl-base-1.0/' \ --train_data_dir='./datasets/online_data \ --original_image_column='input_image' \ --edited_image_column='file_name' \ --edit_prompt_column='prompts' \ --use_ema \ --resolution=512 --random_flip \ --train_batch_size=4 --gradient_accumulation_steps=4 --gradient_checkpointing \ --max_train_steps=15000 \ --checkpointing_steps=5000 --checkpoints_total_limit=1 \ --learning_rate=5e-05 --lr_warmup_steps=0 \ --conditioning_dropout_prob=0.05 \ --seed=42 \ --val_image_url_or_path='./datasets/emoji_online_data/6110789input.jpg' \ --validation_prompt="women, angry" \ --output_dir="./sd_models/lora_models/img2img_test1" \

and recived following error: image at 3100006input.jpg doesn't have metadata in ./datasets/online_data/metadata.jsonl.

It seems i did it wrong in the dataset format, what should i do?

[sayakpaul]

@lhoestq could you suggest @DoctorTar how the local dataset should be structured here?

[lhoestq]

Not sure but if train_instruct_pix2pix_sdxl.py uses datasets you probably need to rename the "input_image" field "file_name" (see the docs at https://huggingface.co/docs/datasets/image_dataset#imagefolder)