search

huggingface / diffusers

🤗 Diffusers: State-of-the-art diffusion models for image and audio generation in PyTorch and FLAX.
https://huggingface.co/docs/diffusers
Apache License 2.0
24.21k stars 4.99k forks source link

VAE config mismatch when using StableDiffusionInpainting #7334

Open sayakpaul opened 4 months ago

sayakpaul commented 4 months ago

Currently, we have got a failing test:

tests/pipelines/stable_diffusion/test_stable_diffusion_inpaint.py::StableDiffusionInpaintPipelineSlowTests::test_single_file_component_configs

And it fails for the right reason! You will notice that the assertion doesn't pass for sample_size. For the diffusers checkpoint, this value is 256 and the inferred value from the single-file checkpoint is 512.

This is a bit of an issue because the sample_size param in the VAE influences the tiling factors: https://github.com/huggingface/diffusers/blob/5d848ec07c2011d600ce5e5c1aa02a03152aea9b/src/diffusers/models/autoencoders/autoencoder_kl.py#L119

So, for the VAE loaded from the diffusers checkpoint will have a different tiling configuration from the one loaded from the single-file checkpoint.

Cc: @DN6 @yiyixuxu

sayakpaul commented 4 months ago

gentle ping @yiyixuxu

yiyixuxu commented 3 months ago

I think this will not be an issue once we have https://github.com/huggingface/diffusers/pull/7496, no?

sayakpaul commented 3 months ago

I will let @DN6 test that.

github-actions[bot] commented 2 months ago

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.