Reproduce Object Erasing Results

[jasper0314]

Hello,

Thank you for your outstanding work on this project. I've been attempting to reproduce the results of object erasing as described in your paper. However, I've encountered an issue where after erasing an object (e.g., "Garbage Truck"), the generated image turns into complete noise regardless of the input prompt.

I used the following commands for training and evaluation with your codebase:

# For training
python train-scripts/train_erase.py \
  --concepts 'Garbage Truck' \
  --guided_concept ' ' \
  --device '0' \
  --concept_type 'object' \
  --add_prompts True

# For evaluation
python eval-scripts/generate-images.py \
  --model_name="$model_name" \
  --prompts_path 'test.csv' \
  --save_path 'evaluation_folder' \
  --num_samples 5 \
  --ddim_steps 50

Here is the content of my test.csv:

,prompt,evaluation_seed,case_number
0,"a photo of garbage truck",3016,0
1,"a photo of bird",3016,1
2,"a photo of woman",3016,2

Could you please help me identify any misunderstandings in the reproduction steps? I appreciate your assistance in resolving this issue.

Thank you very much!

[rohitgandikota]

Hey,

I think this is some issue with diffusers update. Could you check the following?

1. check the diffusers version
2. check the parameters that you are training - print(ca_layers) in the edit_model
3. can you try removing the --add_prompts argument while training?

[rohitgandikota]

Hi @jasper0314 , we are looking into this issue. Will notify once we release the fix

[rohitgandikota]

Okay so we tried it from our end and these are the results for "garbage truck" erasure.

original model

edited Model

train-script python train-scripts/train_erase.py --concepts 'Garbage Truck' --guided_concept ' ' --device '0' --concept_type 'object'

eval-script: python eval-scripts/generate-images.py --model_name 'erased-garbage truck-towards_-preserve_false-sd_1_4-method_replace.pt' --prompts_path 'data/garbage_truck.csv' --save_path '/share/u/rohit/www/uce_test'

[rohitgandikota]

We use diffusers version 0.20.2 in our experiments. Could you please try again?

[rohitgandikota]

@jasper0314, I think it has something to do with the library version. We uploaded the requirements.txt to the repo now. Could you try installing the packages using pip install -r requirements.txt and try again?

I am sure this would fix the issue. If it doesn't, please feel free to reopen the issue.

[jasper0314]

@rohitgandikota Sorry for the late reply. I follow those commands you provided and solve the problems. Thanks!