How to use fp16 precison version of Stable Diffusion 1.5?

[felixslu]

when I use fp16 version ,I got an capture error on my 3090TI.

pipe = StableDiffusionPipeline.from_pretrained( "runwayml/stable-diffusion-v1-5", revision='fp16', torch_dtype=torch.float16, local_files_only=True )

RuntimeError: "LayerNormKernelImpl" not implemented for 'Half'

[felixslu]

TVMError: Data types float32 and float16 must be equal for binary operators

[10:32:31] ~/tvm/tvm/src/relax/ir/block_builder.cc:64: Warning: BlockBuilder destroyed with remaining blocks! [2023-07-13 10:32:31,987] torch._dynamo.convert_frame: [ERROR] WON'T CONVERT forward /root/miniconda3/envs/tvm-build/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py line 363 due to: Traceback (most recent call last): File "~/tvm/tvm/python/tvm/_ffi/_ctypes/packed_func.py", line 238, in call raise get_last_ffi_error() torch._dynamo.exc.BackendCompilerFailed: backend='_capture' raised: TVMError: Traceback (most recent call last): 12: TVMFuncCall 1: tvm::relax::InferBinaryArithOpOutDtype(tvm::relax::Call const&, tvm::relax::BlockBuilder const&, tvm::relax::TensorStructInfo const&, tvm::relax::TensorStructInfo const&) 0: tvm::relax::BlockBuilderImpl::ReportFatal(tvm::Diagnostic const&) File "/home/luting6/car/tvm/tvm/src/relax/ir/block_builder.cc", line 138 TVMError: Data types float32 and float16 must be equal for binary operators

Traceback (most recent call last): File "build.py", line 280, in mod, params = trace_models(torch_dev_key) File "build.py", line 144, in trace_models clip = trace.clip_to_text_embeddings(pipe) File "~/web-stable-diffusion/web_stable_diffusion/trace/model_trace.py", line 30, in clip_to_text_embeddings mod = dynamo_capture_subgraphs( File "~/tvm/tvm/python/tvm/relax/frontend/torch/dynamo.py", line 165, in dynamo_capture_subgraphs compiled_model(*params, kwargs) File "/root/miniconda3/envs/tvm-build/lib/python3.8/site-packages/torch/_dynamo/eval_frame.py", line 294, in _fn return fn(*args, *kwargs) File "~/web-stable-diffusion/web_stable_diffusion/trace/model_trace.py", line 22, in forward text_embeddings = self.clip(text_input_ids)[0] File "/root/miniconda3/envs/tvm-build/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1522, in _wrapped_call_impl return self._call_impl(args, kwargs) File "/root/miniconda3/envs/tvm-build/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1531, in _call_impl return forward_call(*args, kwargs) File "/root/miniconda3/envs/tvm-build/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py", line 816, in forward return self.text_model( File "/root/miniconda3/envs/tvm-build/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1522, in _wrapped_call_impl return self._call_impl(*args, *kwargs) File "/root/miniconda3/envs/tvm-build/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1531, in _call_impl return forward_call(args, kwargs) File "/root/miniconda3/envs/tvm-build/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py", line 725, in forward encoder_outputs = self.encoder( File "/root/miniconda3/envs/tvm-build/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1522, in _wrapped_call_impl return self._call_impl(*args, kwargs) File "/root/miniconda3/envs/tvm-build/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1531, in _call_impl return forward_call(*args, *kwargs) File "/root/miniconda3/envs/tvm-build/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py", line 654, in forward layer_outputs = encoder_layer( File "/root/miniconda3/envs/tvm-build/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1522, in _wrapped_call_impl return self._call_impl(args, kwargs) File "/root/miniconda3/envs/tvm-build/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1531, in _call_impl return forward_call(*args, kwargs) File "/root/miniconda3/envs/tvm-build/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py", line 382, in forward hidden_states = self.layer_norm1(hidden_states) File "/root/miniconda3/envs/tvm-build/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1522, in _wrapped_call_impl return self._call_impl(*args, *kwargs) File "/root/miniconda3/envs/tvm-build/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1531, in _call_impl return forward_call(args, kwargs) File "/root/miniconda3/envs/tvm-build/lib/python3.8/site-packages/torch/nn/modules/normalization.py", line 190, in forward return F.layer_norm( File "/root/miniconda3/envs/tvm-build/lib/python3.8/site-packages/torch/nn/functional.py", line 2548, in layer_norm return torch.layer_norm(input, normalized_shape, weight, bias, eps, torch.backends.cudnn.enabled) RuntimeError: "LayerNormKernelImpl" not implemented for 'Half'