xiaoyu-work
commented
1 month ago @PatriceVignola can you please take a look?
Open fragfeaster777 opened 1 month ago
xiaoyu-work
commented
1 month ago @PatriceVignola can you please take a look?
DominicSYHsieh
commented
1 month ago I have some almost the same issues, here's the error log :
Optimizing text_encoder
[2024-08-08 02:36:04,730] [INFO] [run.py:138:run_engine] Running workflow default_workflow
[2024-08-08 02:36:04,733] [INFO] [engine.py:986:save_olive_config] Saved Olive config to cache\default_workflow\olive_config.json
[2024-08-08 02:36:04,733] [INFO] [accelerator_creator.py:224:create_accelerators] Running workflow on accelerator specs: gpu-dml
[2024-08-08 02:36:04,733] [INFO] [engine.py:109:initialize] Using cache directory: cache\default_workflow
[2024-08-08 02:36:04,735] [INFO] [engine.py:265:run] Running Olive on accelerator: gpu-dml
[2024-08-08 02:36:04,735] [INFO] [engine.py:1085:_create_system] Creating target system ...
[2024-08-08 02:36:04,735] [INFO] [engine.py:1088:_create_system] Target system created in 0.000000 seconds
[2024-08-08 02:36:04,735] [INFO] [engine.py:1097:_create_system] Creating host system ...
[2024-08-08 02:36:04,735] [INFO] [engine.py:1100:_create_system] Host system created in 0.000000 seconds
[2024-08-08 02:36:04,773] [INFO] [engine.py:867:_run_pass] Running pass convert:OnnxConversion
Traceback (most recent call last):
File "C:\Users\Acer\Desktop\Olive\examples\directml\stable_diffusion_xl\stable_diffusion_xl.py", line 635, in
Invoked with: %341 : Tensor = onnx::Constant(), scope: transformers.models.clip.modeling_clip.CLIPTextModel::/transformers.models.clip.modeling_clip.CLIPTextTransformer::text_model/transformers.models.clip.modeling_clip.CLIPEncoder::encoder/transformers.models.clip.modeling_clip.CLIPEncoderLayer::layers.0/transformers.models.clip.modeling_clip.CLIPSdpaAttention::self_attn , 'value', 0.125 (Occurred when translating scaled_dot_product_attention).
as well as the text_encoder_2, error log:
Optimizing text_encoder_2
[2024-08-08 02:24:50,086] [INFO] [run.py:138:run_engine] Running workflow default_workflow
[2024-08-08 02:24:50,089] [INFO] [engine.py:986:save_olive_config] Saved Olive config to cache\default_workflow\olive_config.json
[2024-08-08 02:24:50,090] [INFO] [accelerator_creator.py:224:create_accelerators] Running workflow on accelerator specs: gpu-dml
[2024-08-08 02:24:50,090] [INFO] [engine.py:109:initialize] Using cache directory: cache\default_workflow
[2024-08-08 02:24:50,090] [INFO] [engine.py:265:run] Running Olive on accelerator: gpu-dml
[2024-08-08 02:24:50,090] [INFO] [engine.py:1085:_create_system] Creating target system ...
[2024-08-08 02:24:50,090] [INFO] [engine.py:1088:_create_system] Target system created in 0.000000 seconds
[2024-08-08 02:24:50,090] [INFO] [engine.py:1097:_create_system] Creating host system ...
[2024-08-08 02:24:50,090] [INFO] [engine.py:1100:_create_system] Host system created in 0.000000 seconds
[2024-08-08 02:24:50,136] [INFO] [engine.py:867:_run_pass] Running pass convert:OnnxConversion
Traceback (most recent call last):
File "C:\Users\Acer\Desktop\Olive\examples\directml\stable_diffusion_xl\stable_diffusion_xl.py", line 635, in
Invoked with: %662 : Tensor = onnx::Constant(), scope: transformers.models.clip.modeling_clip.CLIPTextModelWithProjection::/transformers.models.clip.modeling_clip.CLIPTextTransformer::text_model/transformers.models.clip.modeling_clip.CLIPEncoder::encoder/transformers.models.clip.modeling_clip.CLIPEncoderLayer::layers.0/transformers.models.clip.modeling_clip.CLIPSdpaAttention::self_attn , 'value', 0.125 (Occurred when translating scaled_dot_product_attention).
System information : OS: Windows 11 Olive : 0.6.2 onnxruntime-directml : tried with 1.18.0 / 1.18.1
Jay19751103
commented
1 month ago Use following can optimize the model, it needs to change json accelerate==0.33.0 aiohappyeyeballs==2.3.4 aiohttp==3.10.1 aiosignal==1.3.1 alembic==1.13.2 annotated-types==0.7.0 async-timeout==4.0.3 attrs==24.2.0 certifi==2024.7.4 charset-normalizer==3.3.2 colorama==0.4.6 coloredlogs==15.0.1 colorlog==6.8.2 config==0.5.1 datasets==2.20.0 diffusers==0.29.2 dill==0.3.8 filelock==3.15.4 flatbuffers==24.3.25 frozenlist==1.4.1 fsspec==2024.5.0 greenlet==3.0.3 huggingface-hub==0.24.5 humanfriendly==10.0 idna==3.7 importlib_metadata==8.2.0 invisible-watermark==0.2.0 Jinja2==3.1.4 lightning-utilities==0.11.6 Mako==1.3.5 MarkupSafe==2.1.5 mpmath==1.3.0 multidict==6.0.5 multiprocess==0.70.16 networkx==3.3 numpy==1.26.4 olive-ai==0.6.2 onnx==1.16.2 onnx-tool==0.9.0 onnx_opcounter==0.0.3 onnxruntime==1.18.0 onnxruntime-directml==1.18.0 opencv-python==4.10.0.84 optimum==1.21.3 optuna==3.6.1 packaging==24.1 pandas==2.2.2 pillow==10.4.0 protobuf==3.20.3 psutil==6.0.0 pyarrow==17.0.0 pyarrow-hotfix==0.6 pydantic==2.7.0 pydantic_core==2.18.1 pyreadline3==3.4.1 python-dateutil==2.9.0.post0 pytz==2024.1 PyWavelets==1.6.0 PyYAML==6.0.1 regex==2024.7.24 requests==2.32.3 safetensors==0.4.4 sentencepiece==0.2.0 six==1.16.0 SQLAlchemy==2.0.32 sympy==1.13.1 tabulate==0.9.0 tokenizers==0.19.1 torch==2.4.0 torchmetrics==1.4.1 tqdm==4.66.5 transformers==4.42.4 typing_extensions==4.12.2 tzdata==2024.1 urllib3==2.2.2 xxhash==3.4.1 yarl==1.9.4 zipp==3.19.2
OLIVE modifications 0001-Fix-optimization-sdxl-issues.patch
Note that weights.pb does not copied into optimized unet folder find in from cache or footprint directory , the size close 5GB and copy it to the folder manually, then you can run it locally.
Jay19751103
commented
3 weeks ago Key for this issue is downgrade transformer to transformers==4.42.4
saddam213
commented
3 weeks ago I am also stuck on this issue
TypeError: z_(): incompatible function arguments. The following argument types are supported:
1. (self: torch._C.Node, arg0: str, arg1: torch.Tensor) -> torch._C.Node
Invoked with: %329 : Tensor = onnx::Constant(), scope: transformers.models.clip.modeling_clip.CLIPTextModel::/transformers.models.clip.modeling_clip.CLIPTextTransformer::text_model/transformers.models.clip.modeling_clip.CLIPEncoder::encoder/transformers.models.clip.modeling_clip.CLIPEncoderLayer::layers.0/transformers.models.clip.modeling_clip.CLIPSdpaAttention::self_attn
, 'value', 0.125
(Occurred when translating scaled_dot_product_attention).This has broken all stable-diffusion converter scripts
DominicSYHsieh
commented
3 weeks ago For this issue, downgrade transformers to 4.42.4 is work for me.
When attempting: python stablediffusion.py --optimize, I get a "TypeError: z(): incompatible function arguments" error for "Optimizing text_encoder". Note that "Optimizing vae_encoder", "Optimizing vae_decoder" and "Optimizing unet" worked.
Log under "Optimizing text_encoder":
[DEBUG] [olive_evaluator.py:1153:validate_metrics] No priority is specified, but only one sub type metric is specified. Use rank 1 for single for this metric. [INFO] [run.py:138:run_engine] Running workflow default_workflow [INFO] [engine.py:986:save_olive_config] Saved Olive config to cache\default_workflow\olive_config.json [DEBUG] [run.py:179:run_engine] Registering pass OnnxConversion [DEBUG] [run.py:179:run_engine] Registering pass OrtTransformersOptimization [DEBUG] [accelerator_creator.py:130:_fill_accelerators] The accelerator device and execution providers are specified, skipping deduce. [DEBUG] [accelerator_creator.py:169:_check_execution_providers] Supported execution providers for device gpu: ['DmlExecutionProvider', 'CPUExecutionProvider'] [DEBUG] [accelerator_creator.py:199:create_accelerators] Initial accelerators and execution providers: {'gpu': ['DmlExecutionProvider']} [INFO] [accelerator_creator.py:224:create_accelerators] Running workflow on accelerator specs: gpu-dml [DEBUG] [run.py:235:run_engine] Pass OnnxConversion already registered [DEBUG] [run.py:235:run_engine] Pass OpenVINOConversion already registered [DEBUG] [run.py:235:run_engine] Pass OrtTransformersOptimization already registered [DEBUG] [run.py:235:run_engine] Pass OrtTransformersOptimization already registered [INFO] [engine.py:109:initialize] Using cache directory: cache\default_workflow [INFO] [engine.py:265:run] Running Olive on accelerator: gpu-dml [INFO] [engine.py:1085:_create_system] Creating target system ... [DEBUG] [engine.py:1081:create_system] create native OliveSystem SystemType.Local [INFO] [engine.py:1088:_create_system] Target system created in 0.000000 seconds [INFO] [engine.py:1097:_create_system] Creating host system ... [DEBUG] [engine.py:1081:create_system] create native OliveSystem SystemType.Local [INFO] [engine.py:1100:_create_system] Host system created in 0.000000 seconds [DEBUG] [engine.py:711:_cache_model] Cached model bebe0e3c to cache\default_workflow\models\bebe0e3c.json [DEBUG] [engine.py:338:run_accelerator] Running Olive in no-search mode ... [DEBUG] [engine.py:430:run_no_search] Running ['convert', 'optimize'] with no search ... [INFO] [engine.py:867:_run_pass] Running pass convert:OnnxConversion [DEBUG] [resource_path.py:156:create_resource_path] Resource path runwayml/stable-diffusion-v1-5 is inferred to be of type string_name. [DEBUG] [resource_path.py:156:create_resource_path] Resource path user_script.py is inferred to be of type file. [DEBUG] [resource_path.py:156:create_resource_path] Resource path runwayml/stable-diffusion-v1-5 is inferred to be of type string_name. [DEBUG] [resource_path.py:156:create_resource_path] Resource path user_script.py is inferred to be of type file. [DEBUG] [resource_path.py:156:create_resource_path] Resource path C:\Users\ih\sd-test\converter\olive\examples\stable_diffusion\user_script.py is inferred to be of type file. [DEBUG] [dummy_inputs.py:45:get_dummy_inputs] Using dummy_inputs_func to get dummy inputs [DEBUG] [conversion.py:234:_export_pytorch_model] Converting model on device cpu with dtype None. Traceback (most recent call last): File "C:\Users\ih\sd-test\converter\olive\examples\stable_diffusion\stable_diffusion.py", line 433, in
main()
File "C:\Users\ih\sd-test\converter\olive\examples\stable_diffusion\stable_diffusion.py", line 370, in main
optimize(common_args.model_id, common_args.provider, unoptimized_model_dir, optimized_model_dir)
File "C:\Users\ih\sd-test\converter\olive\examples\stable_diffusion\stable_diffusion.py", line 244, in optimize
run_res = olive_run(olive_config)
File "C:\Users\ih.conda\envs\converter\lib\site-packages\olive\workflows\run\run.py", line 297, in run
return run_engine(package_config, run_config, data_root)
File "C:\Users\ih.conda\envs\converter\lib\site-packages\olive\workflows\run\run.py", line 261, in run_engine
engine.run(
File "C:\Users\ih.conda\envs\converter\lib\site-packages\olive\engine\engine.py", line 267, in run
run_result = self.run_accelerator(
File "C:\Users\ih.conda\envs\converter\lib\site-packages\olive\engine\engine.py", line 339, in run_accelerator
output_footprint = self.run_no_search(
File "C:\Users\ih.conda\envs\converter\lib\site-packages\olive\engine\engine.py", line 431, in run_no_search
should_prune, signal, model_ids = self._run_passes(
File "C:\Users\ih.conda\envs\converter\lib\site-packages\olive\engine\engine.py", line 829, in _run_passes
model_config, model_id = self._run_pass(
File "C:\Users\ih.conda\envs\converter\lib\site-packages\olive\engine\engine.py", line 937, in _run_pass
output_model_config = host.run_pass(p, input_model_config, data_root, output_model_path, pass_search_point)
File "C:\Users\ih.conda\envs\converter\lib\site-packages\olive\systems\local.py", line 32, in run_pass
output_model = the_pass.run(model, data_root, output_model_path, point)
File "C:\Users\ih.conda\envs\converter\lib\site-packages\olive\passes\olive_pass.py", line 224, in run
output_model = self._run_for_config(model, data_root, config, output_model_path)
File "C:\Users\ih.conda\envs\converter\lib\site-packages\olive\passes\onnx\conversion.py", line 132, in _run_for_config
output_model = self._run_for_config_internal(model, data_root, config, output_model_path)
File "C:\Users\ih.conda\envs\converter\lib\site-packages\olive\passes\onnx\conversion.py", line 182, in _run_for_config_internal
return self._convert_model_on_device(model, data_root, config, output_model_path, device, torch_dtype)
File "C:\Users\ih.conda\envs\converter\lib\site-packages\olive\passes\onnx\conversion.py", line 439, in _convert_model_on_device
converted_onnx_model = OnnxConversion._export_pytorch_model(
File "C:\Users\ih.conda\envs\converter\lib\site-packages\torch\utils_contextlib.py", line 116, in decorate_context
return func(*args, kwargs)
File "C:\Users\ih.conda\envs\converter\lib\site-packages\olive\passes\onnx\conversion.py", line 285, in _export_pytorch_model
torch.onnx.export(
File "C:\Users\ih.conda\envs\converter\lib\site-packages\torch\onnx\utils.py", line 551, in export
_export(
File "C:\Users\ih.conda\envs\converter\lib\site-packages\torch\onnx\utils.py", line 1648, in _export
graph, params_dict, torch_out = _model_to_graph(
File "C:\Users\ih.conda\envs\converter\lib\site-packages\torch\onnx\utils.py", line 1174, in _model_to_graph
graph = _optimize_graph(
File "C:\Users\ih.conda\envs\converter\lib\site-packages\torch\onnx\utils.py", line 714, in _optimize_graph
graph = _C._jit_pass_onnx(graph, operator_export_type)
File "C:\Users\ih.conda\envs\converter\lib\site-packages\torch\onnx\utils.py", line 1997, in _run_symbolic_function
return symbolic_fn(graph_context, *inputs, *attrs)
File "C:\Users\ih.conda\envs\converter\lib\site-packages\torch\onnx\symbolic_helper.py", line 292, in wrapper
return fn(g, args, kwargs)
File "C:\Users\ih.conda\envs\converter\lib\site-packages\torch\onnx\symbolic_opset14.py", line 177, in scaled_dot_product_attention
query_scaled = g.op("Mul", query, g.op("Sqrt", scale))
File "C:\Users\ih.conda\envs\converter\lib\site-packages\torch\onnx_internal\jit_utils.py", line 93, in op
return _add_op(self, opname, *raw_args, outputs=outputs, **kwargs)
File "C:\Users\ih.conda\envs\converter\lib\site-packages\torch\onnx_internal\jit_utils.py", line 244, in _add_op
inputs = [_const_if_tensor(graph_context, arg) for arg in args]
File "C:\Users\ih.conda\envs\converter\lib\site-packages\torch\onnx_internal\jit_utils.py", line 244, in
inputs = [_const_if_tensor(graph_context, arg) for arg in args]
File "C:\Users\ih.conda\envs\converter\lib\site-packages\torch\onnx_internal\jit_utils.py", line 276, in _const_if_tensor
return _add_op(graph_context, "onnx::Constant", value_z=arg)
File "C:\Users\ih.conda\envs\converter\lib\site-packages\torch\onnx_internal\jit_utils.py", line 252, in _add_op
node = _create_node(
File "C:\Users\ih.conda\envs\converter\lib\site-packages\torch\onnx_internal\jit_utils.py", line 312, in _create_node
_add_attribute(node, key, value, aten=aten)
File "C:\Users\ih.conda\envs\converter\lib\site-packages\torch\onnx_internal\jit_utils.py", line 363, in _addattribute
return getattr(node, f"{kind}")(name, value)
TypeError: z_(): incompatible function arguments. The following argument types are supported:
Invoked with: %340 : Tensor = onnx::Constant(), scope: transformers.models.clip.modeling_clip.CLIPTextModel::/transformers.models.clip.modeling_clip.CLIPTextTransformer::text_model/transformers.models.clip.modeling_clip.CLIPEncoder::encoder/transformers.models.clip.modeling_clip.CLIPEncoderLayer::layers.0/transformers.models.clip.modeling_clip.CLIPSdpaAttention::self_attn , 'value', 0.125 (Occurred when translating scaled_dot_product_attention).
Other information