search

megvii-research / HiDiffusion

[ECCV 2024] HiDiffusion: Increases the resolution and speed of your diffusion model by only adding a single line of code!
Apache License 2.0
731 stars 38 forks source link

hidiffusion.py fixes #3

Open JohnnyStreet opened 4 months ago

JohnnyStreet commented 4 months ago

You define the name_or_path where model.name_or_path is not known but then you reference model.name_or_path anyway which breaks it for non-default models.

Line 1831:

'is_inpainting_task': 'inpainting' in model.name_or_path, 'is_playground': 'playground' in model.name_or_path}

Should be:

'is_inpainting_task': 'inpainting' in name_or_path, 'is_playground': 'playground' in name_or_path}

You also have a typo in a variable name which should be supported_official_model

It is working with this fix although I did notice you do not get the same image when you use the same seed twice. Not sure if this is expected behavior or not.

ShenZhang-Shin commented 4 months ago

Thanks a lot. "name_or_path" is the backbone model being used, and "model.name_or_path" refers to the name being passed in. 'inpainting' in name_or_path is right, but 'is_playground': 'playground' in name_or_path is not equivalent to 'playground' in model.name_or_path. It is a bit convoluted. I will streamline it. typo will also be fixed.

JohnnyStreet commented 4 months ago

"name_or_path" is the backbone model being used, and "model.name_or_path" refers to the name being passed in.

There was no model.name_or_path on my model so it was referencing a NoneType. I made that change and it seems to work. I can confirm it is doing something by using the same seed as without. The results on widescreen images are much more coherent, without the usual repeating or stretching associated with it trying to tile.

vladmandic commented 4 months ago

@ShenZhang-Shin first, diffusers dont have name_or_path param, so that will always be invalid. also, using model name to determine if its inpainting task is flawed to start with.

@JohnnyStreet your proposed solution will always result in false, so inpainting mode will not be detected

you can use this:

  from diffusers.pipelines import auto_pipeline
  is_inpainting_task = model.__class__ in auto_pipeline.AUTO_INPAINT_PIPELINES_MAPPING.values()