无法得到个性化的生成结果

JoshonSmith commented 2 months ago

根据开源的训练脚本和数据样例，无法得到个性化的生成结果

训练脚本：scripts/train_single.sh 训练数据：data/dog

训练完后，测试代码：

import torch
from diffusers import DiffusionPipeline

pipeline = DiffusionPipeline.from_pretrained(
    "runwayml/stable-diffusion-v1-5", torch_dtype=torch.float16,
).to("cuda")
pipeline.unet.load_attn_procs("ckpt路径", weight_name="pytorch_custom_diffusion_weights.bin")
pipeline.load_textual_inversion("ckpt路径", <new1>.bin")

image = pipeline(
    "<new1> dog swimming in the pool",
    num_inference_steps=100,
    guidance_scale=6.0,
    eta=1.0,
).images[0]
image.save("dog.png")

生成结果与训练集中的dog 完全不一致： dog1 dog

Rbrq03 commented 2 months ago

Hello @JoshonSmith, thanks for interest. You should check whether PEFT is installed in your environment, as it results in a bug. For more detailed, you can check PR#7272, Issue#7261. We are moving forward with this PR merge. If you continue to experience issues, feel free to reach out.

JoshonSmith commented 2 months ago

Hello @JoshonSmith, thanks for interest. You should check whether PEFT is installed in your environment, as it results in a bug. For more detailed, you can check PR#7272, Issue#7261. We are moving forward with this PR merge. If you continue to experience issues, feel free to reach out.

I install this diffusers:

https://github.com/Rbrq03/diffusers/tree/custom_diffusion_peft_fix this error is solved

Rbrq03 / ClassDiffusion

无法得到个性化的生成结果 #3