facebookincubator / AITemplate

AITemplate is a Python framework which renders neural network into high performance CUDA/HIP C++ code. Specialized for FP16 TensorCore (NVIDIA GPU) and MatrixCore (AMD GPU) inference.
Apache License 2.0
4.54k stars 363 forks source link

<class 'src.pipeline_stable_diffusion_ait.StableDiffusionAITPipeline'> is incorrectly implemented. Expected {'feature_extractor', 'scheduler', 'tokenizer', 'text_encoder', 'safety_checker', 'unet', 'vae'} to be defined #975

Open jiagaoxiang opened 10 months ago

jiagaoxiang commented 10 months ago

Run this line of code: python3 scripts/demo.py --prompt "Mountain Rainier in van Gogh's world"

Got the following errors:

[20:11:19] model_container.cpp:87: Init AITemplate Runtime with 1 concurrency Traceback (most recent call last): File "/home/aac/AITemplate/examples/05_stable_diffusion/scripts/demo.py", line 66, in run() File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/click/core.py", line 1157, in call return self.main(args, kwargs) File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/click/core.py", line 1078, in main rv = self.invoke(ctx) File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/click/core.py", line 1434, in invoke return ctx.invoke(self.callback, ctx.params) File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/click/core.py", line 783, in invoke return __callback(args, **kwargs) File "/home/aac/AITemplate/examples/05_stable_diffusion/scripts/demo.py", line 51, in run ).to("cuda") File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/diffusers/pipelines/pipeline_utils.py", line 832, in to module_is_sequentiallyoffloaded(module) for , module in self.components.items() File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/diffusers/pipelines/pipeline_utils.py", line 1954, in components raise ValueError( ValueError: StableDiffusionAITPipeline { "_class_name": "StableDiffusionAITPipeline", "_diffusers_version": "0.24.0", "_name_or_path": "./tmp/diffusers-pipeline/stabilityai/stable-diffusion-v2", "feature_extractor": [ "transformers", "CLIPFeatureExtractor" ], "image_encoder": [ null, null ], "requires_safety_checker": true, "safety_checker": [ "stable_diffusion", "StableDiffusionSafetyChecker" ], "scheduler": [ "diffusers", "EulerDiscreteScheduler" ], "text_encoder": [ "transformers", "CLIPTextModel" ], "tokenizer": [ "transformers", "CLIPTokenizer" ], "unet": [ "diffusers", "UNet2DConditionModel" ], "vae": [ "diffusers", "AutoencoderKL" ] } has been incorrectly initialized or <class 'src.pipeline_stable_diffusion_ait.StableDiffusionAITPipeline'> is incorrectly implemented. Expected {'feature_extractor', 'scheduler', 'tokenizer', 'text_encoder', 'safety_checker', 'unet', 'vae'} to be defined, but dict_keys(['vae', 'text_encoder', 'tokenizer', 'unet', 'scheduler', 'safety_checker', 'feature_extractor', 'image_encoder']) are defined.

ipiszy commented 10 months ago

Maybe worth checking whether the huggingface packages that you use are supported by AIT. AIT SD support hasn't been updated for a while. https://github.com/facebookincubator/AITemplate/tree/main/examples/05_stable_diffusion.

jiagaoxiang commented 10 months ago

Maybe worth checking whether the huggingface packages that you use are supported by AIT. AIT SD support hasn't been updated for a while. https://github.com/facebookincubator/AITemplate/tree/main/examples/05_stable_diffusion.

I am strictly following the AIT tutorial on stable diffusion (on AMD ROCm). Has Meta stopped supporting AIT?

fsx950223 commented 9 months ago

Use this https://github.com/ROCmSoftwarePlatform/AITemplate instead.

mkknightr commented 3 months ago

I got a similar error : And I Fix that by changing my diffusers' version from 0.29 to 0.11.0(which is the version of the official AITemplate repo's).