search

huggingface / optimum-habana

Easy and lightning fast training of 🤗 Transformers on Habana Gaudi processor (HPU)
Apache License 2.0
132 stars 152 forks source link

Info needed about stable diffusion 3 support #1127

Open dkiran1 opened 1 week ago

dkiran1 commented 1 week ago

Feature request

is stabilityai/stable-diffusion-3-medium-diffusers model supported on Gaudi, I tried to run the model , Iam getting the below error ... Traceback (most recent call last): File "/home/app_user/app/stable_diffusion_inference.py", line 9, in import text_to_image_service as ttoi File "/home/app_user/app/text_to_image_service.py", line 107, in pipeline = GaudiStableDiffusionPipeline.from_pretrained( File "/home/app_user/.local/lib/python3.10/site-packages/optimum/habana/diffusers/pipelines/pipeline_utils.py", line 359, in from_pretrained return super().from_pretrained( File "/home/app_user/.local/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 119, in _inner_fn return fn(args, kwargs) File "/home/app_user/.local/lib/python3.10/site-packages/diffusers/pipelines/pipeline_utils.py", line 1286, in from_pretrained loaded_sub_model = load_sub_model( File "/home/app_user/.local/lib/python3.10/site-packages/diffusers/pipelines/pipeline_utils.py", line 531, in load_sub_model loaded_sub_model = load_method(os.path.join(cached_folder, name), loading_kwargs) File "/home/app_user/.local/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 119, in _inner_fn return fn(args, **kwargs) File "/home/app_user/.local/lib/python3.10/site-packages/diffusers/models/modeling_utils.py", line 662, in from_pretrained raise ValueError( ValueError: Cannot load <class 'diffusers.models.autoencoders.autoencoder_kl.AutoencoderKL'> from /home/app_user/.cache/huggingface/hub/models--stabilityai--stable-diffusion-3-medium-diffusers/snapshots/ea42f8cef0f178587cf766dc8129abd379c90671/vae because the following keys are missing: quant_conv.bias, post_quant_conv.bias, post_quant_conv.weight, quant_conv.weight. Please make sure to pass low_cpu_mem_usage=False and device_map=None if you want to randomly initialize those weights or else make sure your checkpoint file is correct.

Motivation

Stable diffusion 3 is the latest model which has additional features

Your contribution

I could send the piece of code which Iam using to test this model

regisss commented 1 week ago

Hi @dkiran1, it's not supposed to work at the moment but will come soon. Please share the code you used here so that we can validate it when the PR is ready.