search

Shilin-LU / MACE

[CVPR 2024] "MACE: Mass Concept Erasure in Diffusion Models" (Official Implementation)
MIT License
272 stars 18 forks source link

Missing evaluation scripts #5

Open LucidStephen opened 2 months ago

LucidStephen commented 2 months ago

Thank you for this solid and interesting work. However, I found that the clip_score evaluation script of the 100_arts style is not included in this repository. The description of the paper about this experiment may result in the two evaluation methods: apply one prompt or five prompts to each art style.

Shilin-LU commented 2 months ago

Hi, thanks for your interest.

The clip_score evaluation script is included in metrics/evaluate_clip_score.py, and you could input the prompt list prompts_csv/art_100_concepts.csv to evaluate the erasure of art style.

We apply 5 prompts to evaluate each art style, and for each prompt we use 5 seeds.

LucidStephen commented 2 months ago

Thank you for your reply. I noticed that sample_images_from_csv.py generates two folders "0" and "1" because of the type attribute in art_100_concepts.csv. This may require some modifications to evaluate_clip_score to accommodate artistic style experiments, such as adding a new args: type.

Shilin-LU commented 2 months ago

Yes, you are right. Just small modifications work well.