search

mlc-ai / web-stable-diffusion

Bringing stable diffusion models to web browsers. Everything runs inside the browser with no server support.
https://mlc.ai/web-stable-diffusion
Apache License 2.0
3.51k stars 220 forks source link

TVMError: Check failed: (!name_supply_->ContainsName(global_symbol_value)) is false: IRModule contains duplicate global symbol: main #62

Closed qiyei2015 closed 2 months ago

qiyei2015 commented 4 months ago

hello, everyone. I follow up get start of https://github.com/mlc-ai/web-stable-diffusion,and run : web-stable-diffusion git:(main) python build.py and Report the following error

Automatically configuring target: metal -keys=metal,gpu -max_function_args=31 -max_num_threads=256 -max_shared_memory_per_block=32768 -max_threads_per_block=1024 -thread_warp_size=32
Load cached module from dist/mod_cache_before_build.pkl and skip tracing. You can use --use-cache=0 to retrace
[13:09:57] /Users/qiyei2016/work/AI/tvm/src/relax/transform/meta_schedule.cc:166: Warning: Tuning record is not found for primfunc: matmul2
[13:09:57] /Users/qiyei2016/work/AI/tvm/src/relax/transform/meta_schedule.cc:166: Warning: Tuning record is not found for primfunc: fused_matmul1_multiply1
[13:09:57] /Users/qiyei2016/work/AI/tvm/src/relax/transform/meta_schedule.cc:166: Warning: Tuning record is not found for primfunc: fused_transpose5_reshape4
[13:09:57] /Users/qiyei2016/work/AI/tvm/src/relax/transform/meta_schedule.cc:166: Warning: Tuning record is not found for primfunc: fused_reshape3_transpose3_transpose4
[13:09:57] /Users/qiyei2016/work/AI/tvm/src/relax/transform/meta_schedule.cc:166: Warning: Tuning record is not found for primfunc: fused_reshape2_transpose_transpose1
[13:09:57] /Users/qiyei2016/work/AI/tvm/src/relax/transform/meta_schedule.cc:166: Warning: Tuning record is not found for primfunc: group_norm1
[13:09:57] /Users/qiyei2016/work/AI/tvm/src/relax/transform/meta_schedule.cc:166: Warning: Tuning record is not found for primfunc: fused_conv2d2_add1_add2_divide_divide
[13:09:58] /Users/qiyei2016/work/AI/tvm/src/relax/transform/meta_schedule.cc:166: Warning: Tuning record is not found for primfunc: fused_reshape3_transpose3
[13:09:58] /Users/qiyei2016/work/AI/tvm/src/relax/transform/meta_schedule.cc:166: Warning: Tuning record is not found for primfunc: softmax
[13:09:58] /Users/qiyei2016/work/AI/tvm/src/relax/transform/meta_schedule.cc:166: Warning: Tuning record is not found for primfunc: transpose
Traceback (most recent call last):
  File "/Users/qiyei2016/work/AI/web-stable-diffusion/build.py", line 169, in <module>
    build(mod, ARGS)
  File "/Users/qiyei2016/work/AI/web-stable-diffusion/build.py", line 137, in build
    ex = relax.build(mod_deploy, args.target)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/qiyei2016/work/AI/tvm/python/tvm/relax/vm_build.py", line 335, in build
    mod = pipeline(mod)
          ^^^^^^^^^^^^^
  File "/Users/qiyei2016/work/AI/tvm/python/tvm/ir/transform.py", line 238, in __call__
    return _ffi_transform_api.RunPass(self, mod)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/qiyei2016/work/AI/tvm/python/tvm/_ffi/_ctypes/packed_func.py", line 239, in __call__
    raise_last_ffi_error()
  File "/Users/qiyei2016/work/AI/tvm/python/tvm/_ffi/base.py", line 481, in raise_last_ffi_error
    raise py_err
  File "/Users/qiyei2016/work/AI/tvm/python/tvm/relax/pipeline.py", line 99, in _pipeline
    mod = seq(mod)
          ^^^^^^^^
  File "/Users/qiyei2016/work/AI/tvm/python/tvm/ir/transform.py", line 238, in __call__
    return _ffi_transform_api.RunPass(self, mod)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/qiyei2016/work/AI/tvm/python/tvm/ir/transform.py", line 307, in _pass_func
    return inst.transform_module(mod, ctx)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/qiyei2016/work/AI/tvm/python/tvm/relax/backend/dispatch_sort_scan.py", line 163, in transform_module
    return sort_scan_dispater.builder_.finalize()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/qiyei2016/work/AI/tvm/python/tvm/relax/block_builder.py", line 682, in finalize
    return _ffi_api.BlockBuilderFinalize(self)  # type: ignore
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/qiyei2016/work/AI/tvm/python/tvm/_ffi/_ctypes/packed_func.py", line 239, in __call__
    raise_last_ffi_error()
  File "/Users/qiyei2016/work/AI/tvm/python/tvm/_ffi/base.py", line 481, in raise_last_ffi_error
    raise py_err
tvm._ffi.base.TVMError: Traceback (most recent call last):
  File "/Users/qiyei2016/work/AI/tvm/src/relax/transform/normalize.cc", line 234
TVMError: Check failed: (!name_supply_->ContainsName(global_symbol_value)) is false: IRModule contains duplicate global symbol: main

The environment is OS: Macos 14.4 CPU: M1max XCode: 15.4

senlyu163 commented 3 months ago

You can refer to the implementation in this repository: https://github.com/happyme531/RK3588-stable-diffusion-GPU/blob/0f2b0b5ead9068bd726fbf26f13cdd5acfcb9710/build.py#L204

qiyei2015 commented 2 months ago

thank you, I modify build.py

diff --git a/build.py b/build.py
index d0def78..4c23f34 100644
--- a/build.py
+++ b/build.py
@@ -128,10 +128,36 @@ def legalize_and_lift_params(
 def build(mod: tvm.IRModule, args: Dict) -> None:
     from tvm import meta_schedule as ms

-    db = ms.database.create(work_dir=args.db_path)
-    with args.target, db, tvm.transform.PassContext(opt_level=3):
-        mod_deploy = relax.transform.MetaScheduleApplyDatabase(enable_warning=True)(mod)
-
+    # db = ms.database.create(work_dir=args.db_path)
+    # with args.target, db, tvm.transform.PassContext(opt_level=3):
+    #     mod_deploy = relax.transform.MetaScheduleApplyDatabase(enable_warning=True)(mod)
+    mod_deploy = mod
+    print("Applying database 1 =======================")
+    db3 = ms.database.create(work_dir=args.db_path) # For some reason, the softmax2 op run very slow on Mali GPU, so I need to tune it separately
+    with args.target, db3, tvm.transform.PassContext(opt_level=3):
+        mod_deploy = relax.transform.MetaScheduleApplyDatabase(enable_warning=True)(mod_deploy)
+    print("Applying database 2 =======================")
+    db0 = ms.database.create(work_dir=args.db_path) # The clip and unet part of the model
+    with args.target, db0, tvm.transform.PassContext(opt_level=3):
+        mod_deploy = relax.transform.MetaScheduleApplyDatabase(enable_warning=True)(mod_deploy)
+    print("Applying database 3 =======================")
+    db2 = ms.database.create(work_dir=args.db_path) # The vae part of the model (Not tuned very well yet)
+    with args.target, db2, tvm.transform.PassContext(opt_level=3):
+        mod_deploy = relax.transform.MetaScheduleApplyDatabase(enable_warning=True)(mod_deploy)
+    print("Generating missing schedules ==============")  
+    with tvm.target.Target("cuda"):
+        mod_deploy = tvm.tir.transform.DefaultGPUSchedule()(mod_deploy) # for some missing schedules
+
+    for gv, func in mod_deploy.functions.items():
+        try:
+            if func.attrs["global_symbol"] == "main" and func.attrs["num_input"] == 3: # u-net
+                mod_deploy[gv] = func.with_attr("global_symbol", "unet")
+            if func.attrs["global_symbol"] == "main" and func.attrs["num_input"] == 1: # vae
+                mod_deploy[gv] = func.with_attr("global_symbol", "vae")
+            if func.attrs["global_symbol"] == "subgraph_0":
+                mod_deploy[gv] = func.with_attr("global_symbol", "clip")
+        except:
+            pass
     debug_dump_script(mod_deploy, "mod_build_stage.py", args)

     ex = relax.build(mod_deploy, args.target)

and run successful,I test on my M1max ➜ web-stable-diffusion git:(main) ✗ python3 deploy.py --prompt "A photo of an astronaut riding a horse on mars." vocab.json: 961kB [00:01, 920kB/s] merges.txt: 525kB [00:00, 1.91MB/s] special_tokens_map.json: 100%|██████████████████████████████████████████████████████████| 389/389 [00:00<00:00, 680kB/s] tokenizer_config.json: 905B [00:00, 1.80MB/s]
100%|███████████████████████████████████████████████████████████████████████████████████| 20/20 [00:16<00:00, 1.19it/s] Time elapsed: 19.560760736465454 seconds, output saved to dist/example.png

thank you @senlyu163

qiyei2015 commented 2 months ago

Close the issue as follows @senlyu163 suggestion

abacaaaaaa commented 2 months ago

@qiyei2015 May I ask what dependencies you used when configuring this project, especially the versions of key packages such as mlc_ai and torch? Did you install mlc-ai directly using pip3 install mlc-ai-nightly -f https://mlc.ai/wheels, or did you build it from source? If possible, could you please share the environment information with me? Thank you very much for your help.

qiyei2015 commented 2 months ago

我放假后把环境信息发给你在 2024年5月1日，21:25，Liu @.***> 写道： @qiyei2015 May I ask what dependencies you used when configuring this project, especially the versions of key packages such as mlc_ai and torch? Did you install mlc-ai directly using pip3 install mlc-ai-nightly -f https://mlc.ai/wheels, or did you build it from source? If possible, could you please share the environment information with me? Thank you very much for your help.

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: @.***>

abacaaaaaa commented 2 months ago

@qiyei2015 Thank you for your reply. Could you please provide some relevant information about the environment? If there are any installation details, such as changes to certain file contents, please include them as well. Thank you very much.

qiyei2015 commented 2 months ago

@abacaaaaaa 我的环境信息： OS:macos 14.4.1 python：Python 3.11.2

另外我的TVM不是安装的，我是用的源码编译的,按照这个文档，切换到origin/unity分支 https://tvm.apache.org/docs/install/from_source.html 使用这个hash ed3790e979b4f0afa86890d2e597783deb67b85e

不过我刚刚看到没有unity这个分支了，你可以用https://github.com/mlc-ai/relax这个试试

另外，在安装requirements.txt，跟cuda相关的安装不成功，可以不安装

abacaaaaaa commented 2 months ago

@qiyei2015 Thank you for your reply. I tried https://github.com/mlc-ai/relax as you suggested, but encountered a ValueError: At least one GPU backend is expected to be enabled error after installation. My environment : MacOS 14.5 and Python 3.11. Would you mind sharing the files from the previous Unity branch with me, possibly through Google Drive or Baidu Cloud? Thank you.

qiyei2015 commented 2 months ago

@abacaaaaaa 我给你的TVM的环境 链接: https://pan.baidu.com/s/1jvawcxX2JZIi6O5RwDUKKw?pwd=913n 提取码: 913n

qiyei2015 commented 2 months ago

我就直接用的pip 安装的 pip install -r requirements.txt

版本信息如下： ➜ web-stable-diffusion git:(main) ✗ pip list
DEPRECATION: Loading egg at /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/click_plugins-1.1.1-py3.11.egg is deprecated. pip 24.3 will enforce this behaviour change. A possible replacement is to use pip for package installation.. Discussion can be found at https://github.com/pypa/pip/issues/12330 DEPRECATION: Loading egg at /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/shodan-1.31.0-py3.11.egg is deprecated. pip 24.3 will enforce this behaviour change. A possible replacement is to use pip for package installation.. Discussion can be found at https://github.com/pypa/pip/issues/12330 DEPRECATION: Loading egg at /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/tldextract-5.1.1-py3.11.egg is deprecated. pip 24.3 will enforce this behaviour change. A possible replacement is to use pip for package installation.. Discussion can be found at https://github.com/pypa/pip/issues/12330 DEPRECATION: Loading egg at /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/requests_file-2.0.0-py3.11.egg is deprecated. pip 24.3 will enforce this behaviour change. A possible replacement is to use pip for package installation.. Discussion can be found at https://github.com/pypa/pip/issues/12330 Package Version Editable project location

absl-py 2.1.0 accelerate 0.27.2 aiofiles 23.2.1 altair 5.2.0 annotated-types 0.6.0 anyio 4.2.0 appnope 0.1.4 argon2-cffi 23.1.0 argon2-cffi-bindings 21.2.0 arrow 1.3.0 asttokens 2.4.1 astunparse 1.6.3 async-lru 2.0.4 attrs 23.1.0 Babel 2.14.0 beautifulsoup4 4.12.3 bleach 6.1.0 canmatrix 1.0 certifi 2022.12.7 cffi 1.16.0 charset-normalizer 3.3.2 click 8.1.7 click-plugins 1.1.1 click-plugins 1.1.1 cloudpickle 3.0.0 colorama 0.4.6 comm 0.2.2 contourpy 1.2.0 cpm-kernels 1.0.11 cycler 0.12.1 debugpy 1.8.1 decorator 5.1.1 defusedxml 0.7.1 deprecation 2.1.0 diffusers 0.26.3 distro 1.8.0 executing 2.0.1 fastapi 0.109.2 fastjsonschema 2.19.1 ffmpy 0.3.2 filelock 3.13.1 flatbuffers 24.3.25 fonttools 4.49.0 fqdn 1.5.1 fsspec 2023.12.2 future 0.18.3 gast 0.5.4 google-pasta 0.2.0 gradio 4.19.1 gradio_client 0.10.0 grpcio 1.62.1 h11 0.14.0 h5py 3.10.0 httpcore 1.0.3 httpx 0.26.0 huggingface-hub 0.20.3 idna 3.6 importlib_metadata 7.0.2 importlib-resources 6.1.1 ipykernel 6.29.3 ipython 8.22.2 ipython-genutils 0.2.0 isoduration 20.11.0 jedi 0.19.1 Jinja2 3.1.2 json5 0.9.24 jsonpointer 2.4 jsonschema 4.21.1 jsonschema-specifications 2023.12.1 jupyter_client 8.6.1 jupyter_contrib_core 0.4.2 jupyter_contrib_nbextensions 0.7.0 jupyter_core 5.7.2 jupyter-events 0.9.1 jupyter-highlight-selected-word 0.2.0 jupyter-lsp 2.2.4 jupyter-nbextensions-configurator 0.6.3 jupyter_server 2.13.0 jupyter_server_terminals 0.5.3 jupyterlab 4.1.5 jupyterlab_pygments 0.3.0 jupyterlab_server 2.25.4 keras 3.1.1 kiwisolver 1.4.5 latex2mathml 3.77.0 libclang 18.1.1 lxml 5.1.0 Markdown 3.5.2 markdown-it-py 3.0.0 MarkupSafe 2.1.3 matplotlib 3.8.3 matplotlib-inline 0.1.6 mdtex2html 1.3.0 mdurl 0.1.2 mistune 3.0.2 ml-dtypes 0.3.2 mlc-ai-nightly 0.15.dev154 mlc_chat 0.1.dev914+g6cf8c798 /Users/qiyei2016/work/AI/mlc-llm/python mpmath 1.3.0 namex 0.0.7 nbclient 0.10.0 nbconvert 7.16.2 nbformat 5.10.3 nest-asyncio 1.6.0 networkx 3.2.1 notebook 7.1.2 notebook_shim 0.2.4 numpy 1.26.2 opt-einsum 3.3.0 optree 0.11.0 orjson 3.9.14 overrides 7.7.0 packaging 23.2 pandas 2.2.0 pandocfilters 1.5.1 parso 0.8.3 pexpect 4.9.0 pillow 10.2.0 pip 24.0 platformdirs 4.2.0 prettytable 3.10.0 prometheus_client 0.20.0 prompt-toolkit 3.0.43 protobuf 3.20.3 psutil 5.9.7 ptyprocess 0.7.0 pure-eval 0.2.2 pycparser 2.21 pydantic 2.6.1 pydantic_core 2.16.2 pydub 0.25.1 Pygments 2.17.2 pyparsing 3.1.1 python-dateutil 2.8.2 python-json-logger 2.0.7 python-multipart 0.0.9 pytz 2024.1 PyYAML 6.0.1 pyzmq 25.1.2 qai-hub 0.9.0 referencing 0.33.0 regex 2023.12.25 requests 2.31.0 requests-file 2.0.0 requests-file 2.0.0 requests-toolbelt 1.0.0 rfc3339-validator 0.1.4 rfc3986-validator 0.1.1 rich 13.7.0 rpds-py 0.18.0 ruff 0.2.2 safetensors 0.4.2 scipy 1.12.0 selinux 0.3.0 semantic-version 2.10.0 Send2Trash 1.8.2 sentencepiece 0.1.99 setuptools 65.5.0 shellingham 1.5.4 shodan 1.31.0 shodan 1.31.0 shodan 1.31.0 shortuuid 1.0.11 six 1.16.0 sniffio 1.3.0 soupsieve 2.5 stack-data 0.6.3 starlette 0.36.3 sympy 1.12 tensorboard 2.16.2 tensorboard-data-server 0.7.2 tensorflow 2.16.1 tensorflow-io-gcs-filesystem 0.36.0 tensorflow-metal 1.1.0 termcolor 2.4.0 terminado 0.18.1 tiktoken 0.6.0 tinycss2 1.2.1 tldextract 5.1.1 tldextract 5.1.1 tokenizers 0.13.3 tomlkit 0.12.0 toolz 0.12.1 torch 2.2.2 torchaudio 2.2.2 torchvision 0.17.2 tornado 6.4 tqdm 4.66.2 traitlets 5.14.2 transformers 4.27.1 typer 0.9.0 types-python-dateutil 2.9.0.20240316 typing_extensions 4.9.0 tzdata 2024.1 uri-template 1.3.0 urllib3 2.2.1 uvicorn 0.27.1 wcwidth 0.2.13 webcolors 1.13 webencodings 0.5.1 websocket-client 1.7.0 websockets 11.0.3 Werkzeug 3.0.1 wheel 0.43.0 wrapt 1.16.0 xlrd 2.0.1 XlsxWriter 3.1.9 xlwt 1.3.0 zipp 3.18.0

2024年5月11日 11:09，abacaaaaaa @.***> 写道：

@qiyei2015 https://github.com/qiyei2015 Thank you very much for your response. I’ve been using the TVM package you provided and have successfully run python3 build.py. However, when I tried to execute python3 deploy.py, I encountered the following error:

Traceback (most recent call last): File "/Users/b03/Desktop/websd/deploy.py", line 169, in deploy_to_pipeline(ARGS) File "/Users/b03/Desktop/websd/deploy.py", line 159, in deploy_to_pipeline image = pipe(args.prompt, args.negative_prompt) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/b03/Desktop/websd/deploy.py", line 97, in call text_embeddings = self.clip_to_text_embeddings(text_input_ids) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/b03/Desktop/websd/deploy.py", line 58, in wrapped_f return f(*args, params) ^^^^^^^^^^^^^^^^ File "/Users/b03/Desktop/tvm/python/tvm/_ffi/_ctypes/packed_func.py", line 239, in call raise_last_ffi_error() File "/Users/b03/Desktop/tvm/python/tvm/_ffi/base.py", line 481, in raise_last_ffi_error raise py_err tvm.error.InternalError: Traceback (most recent call last): File "/Users/b03/desktop/tvm/src/runtime/relax_vm/vm.cc", line 652 InternalError: Check failed: static_cast(gfunc.num_args) == args.size() (197 vs. 2) : ValueError: Invoking function clip requires 197 inputs but only 2 inputs are provided. Would you happen to have any relevant information on this? If it’s convenient for you, could you please provide detailed requirements, including specific versions and other installation details? I'm genuinely grateful for your assistance.

— Reply to this email directly, view it on GitHub https://github.com/mlc-ai/web-stable-diffusion/issues/62#issuecomment-2105503777, or unsubscribe https://github.com/notifications/unsubscribe-auth/AD42ZLPGQW5DWEEK6CPSOU3ZBWDXHAVCNFSM6AAAAABE24NCVCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMBVGUYDGNZXG4. You are receiving this because you were mentioned.

abacaaaaaa commented 2 months ago

Thank you very much for your help. I have successfully run deploy.py.