Setting seed in StableDiffusionInstructPix2PixPipeline throws error

Marcophono2 commented 1 year ago

Describe the bug

Hello! I tried to use this code to generate a pix2pix image with a given seed:

model_id = "timbrooks/instruct-pix2pix"

prompt = "turn the image into Retro comic style artwork style"
guidance_scale = 7
negative_prompt=""
image_guidance_scale=1.9
num_inference_steps=22
generator = torch.Generator(device='cuda').manual_seed(666)

ddim = DDIMScheduler.from_config("runwayml/stable-diffusion-v1-5", subfolder="scheduler",use_auth_token="hf_")

pipe = StableDiffusionInstructPix2PixPipeline.from_pretrained(model_id,scheduler=ddim, generator=generator, torch_dtype=torch.float16, safety_checker=None).to("cuda")
im=Image.open("/home/marc/Schreibtisch/AI/portrait_marc_k.jpg")

The error message is

Keyword arguments {'generator': <torch._C.Generator object at 0x7f2676519ff0>} are not expected by StableDiffusionInstructPix2PixPipeline and will be ignored.

But on the other hand side I can fing the generator parameter in the pipeline_stable_diffusion_instruct_pix2pix.py

What am I doing wrong? I want to find out if the results are the same for same prompt, seed and all parameters in Automatic 1111 and if using Diffusers. If someone can confirm this I could quit to figure out the seed implementation. ;)

Reproduction

described above

Logs

No response

System Info

Diffusers 0.12.1 Ubuntu 22.04 Python 3.10.6 Torch 2.0.0 xFormers

patrickvonplaten commented 1 year ago

Hey @Marcophono2,

Please note that the generator should be passed through the call method and not the from_pretrained method. See: https://huggingface.co/docs/diffusers/main/en/api/pipelines/stable_diffusion/pix2pix#diffusers.StableDiffusionInstructPix2PixPipeline.__call__.generator

Marcophono2 commented 1 year ago

Okay. Thanks, @patrickvonplaten !

Marcophono2 commented 1 year ago

@patrickvonplaten Closed a moment too early... How does that call method work!? I never saw code with that call method. Are there Python templates available? Or is this per default for command line?

patrickvonplaten commented 1 year ago

The __call__ method is called every time you do:


pipe = StableDiffusionPipeline.from_pretrained(....)

pipe("this is a prompt")

So when you just "call" the pipe instance, Python "redirects" the function arguments automatically to the method defined as __call__. Maybe also see: https://www.geeksforgeeks.org/__call__-in-python/

huggingface / diffusers