[Inference]

[minsu1206]

Hi Thanks for sharing your implementation I met this issue but I am newbie at Diffuser library ... So I want to ask you how to fix this error

Traceback (most recent call last):
  File "/workspace/realfill/infer.py", line 71, in <module>
    results = pipe(
  File "/opt/conda/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/workspace/3DGS-Diff/diffusers/src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion_inpaint.py", line 1346, in __call__
    latent_model_input = torch.cat([latent_model_input, mask, masked_image_latents], dim=1)
RuntimeError: Sizes of tensors must match except in dimension 1. Expected size 64 but got size 512 for tensor number 2 in the list.

Here is my script

export MODEL_NAME="stabilityai/stable-diffusion-2-inpainting"
export TRAIN_DIR="data/flowerwoman"
export OUTPUT_DIR="flowerwoman-model"

accelerate launch train_realfill.py \
  --pretrained_model_name_or_path=$MODEL_NAME \
  --train_data_dir=$TRAIN_DIR \
  --output_dir=$OUTPUT_DIR \
  --resolution=512 \
  --train_batch_size=16 \
  --gradient_accumulation_steps=1 \
  --unet_learning_rate=2e-4 \
  --text_encoder_learning_rate=4e-5 \
  --lr_scheduler="constant" \
  --lr_warmup_steps=100 \
  --max_train_steps=2000 \
  --lora_rank=8 \
  --lora_dropout=0.1 \
  --lora_alpha=16
# There's no problem during training !!
accelerate launch infer.py \
  --model_path flowerwoman-model \
  --output_dir $OUTPUT_DIR \
  --validation_image $TRAIN_DIR/target/target.png \
  --validation_mask $TRAIN_DIR/target/mask.png

Thanks in advance

[minsu1206]

I also passed heigh and width argument at infer.py like below.

results = pipe(
        ["a photo of sks"] * 16, image=image, mask_image=mask_image,
        height=image.height, width=image.width,
        num_inference_steps=200, guidance_scale=1, generator=generator, 
    ).images

But this didn't help (same error)

[dae-sun]

convert image = Image.open(args.validation_image) to image = Image.open(args.validation_image).convert("RGB")

[thuanz123]

Hi @minsu1206, can you try this updated pip requirements to see if such error stil exists ?

[minsu1206]

Sorry for very late reply.

After setting my environment as you pointed, I solved the issue.

Thanks. I'll close the issue.