search

Rbrq03 / ClassDiffusion

ClassDiffusion: Official impl. of Paper "ClassDiffusion: More Aligned Personalization Tuning with Explicit Class Guidance"
34 stars 0 forks source link

Can not reproduce the example #10

Open phamthibichnga opened 6 days ago

phamthibichnga commented 6 days ago

Hi authors, I am attempting to reproduce the results using the provided code and pretrained weight in Google Colab. the code I am using: `import torch from diffusers import DiffusionPipeline

pipeline = DiffusionPipeline.from_pretrained( "runwayml/stable-diffusion-v1-5", torch_dtype=torch.float16, low_cpu_mem_usage=False ).to("cuda")

pipeline.unet.load_attn_procs("/content/ClassDiffusion/ckpt/dog_cls/", weight_name="pytorch_custom_diffusion_weights.bin") pipeline.load_textual_inversion("/content/ClassDiffusion/ckpt/dog_cls/", weight_name=".bin")

image = pipeline( " dog", num_inference_steps=100, guidance_scale=6.0, eta=1.0, ).images[0] image.save("dog.png")`

I used the weights provided in your pretrained example, accessible via this link: [Pretrained Weights(https://drive.google.com/drive/folders/12KhBmFCUb2opotOQeAH0-dvW9XArwukt.). However, the generated results do not resemble the training images demonstrated in your examples. Could you kindly provide guidance on potential issues or adjustments I might need to make?

transformers version: 4.46.2 diffusers version: 0.31.0 accelerate version: 1.1.1

Thank you for your assistance.

dog (1) dog (2) dog (4)

Link notebook: https://drive.google.com/file/d/12RcIB1soHxGvhroa18fiWifWJ_kioBNp/view?usp=sharing

Rbrq03 commented 6 days ago

Hey @phamthibichnga, thanks for your interest in our work!

I believe a similar issue is discussed in https://github.com/Rbrq03/ClassDiffusion/issues/3. A quick fix is to install https://github.com/Rbrq03/diffusers instead of the official diffusers repo. If you are interested in why this happened, you can refer to it.

Feel free to reach us if you have further questions.

Rbrq03 commented 6 days ago

I will check if we can still proceed with merging my PR into the diffusers repository. However, I'm concerned that it may not be possible, as I noticed a significant modification in that section of the code made earlier by the diffusers team. If merging isn't feasible, I will update the requirements.txt file to prevent this issue from occurring again, at least within this repository.

Anyway, thanks for reminding me.