WASasquatch / was-node-suite-comfyui

An extensive node suite for ComfyUI with over 210 new nodes
MIT License
1.19k stars 178 forks source link

issue with installation #101

Open sparky4 opened 1 year ago

sparky4 commented 1 year ago
 [786/1572] Compiling C object scipy/stats/_unuran/unuran_wrapper.cpython-311-x86_64-linux-gnu.so.p/.._..__lib_unuran_unuran_src_distr_corder.c.o
      [787/1572] Compiling C object scipy/stats/_unuran/unuran_wrapper.cpython-311-x86_64-linux-gnu.so.p/.._..__lib_unuran_unuran_src_distr_cont.c.o
      [788/1572] Compiling C object scipy/stats/_unuran/unuran_wrapper.cpython-311-x86_64-linux-gnu.so.p/.._..__lib_unuran_unuran_src_distr_cvemp.c.o
      [789/1572] Compiling C object scipy/stats/_levy_stable/levyst.cpython-311-x86_64-linux-gnu.so.p/meson-generated_levyst.c.o
      [790/1572] Generating scipy/stats/_hypotests_pythran with a custom command
      FAILED: scipy/stats/_hypotests_pythran.cpp
      /tmp/pip-build-env-dlt2pafg/overlay/bin/pythran -E ../../scipy/stats/_hypotests_pythran.py -o scipy/stats/_hypotests_pythran.cpp
      Traceback (most recent call last):
        File "/tmp/pip-build-env-dlt2pafg/overlay/bin/pythran", line 8, in <module>
          sys.exit(run())
                   ^^^^^
        File "/tmp/pip-build-env-dlt2pafg/overlay/lib/python3.11/site-packages/pythran/run.py", line 180, in run
          pythran.compile_pythranfile(args.input_file,
        File "/tmp/pip-build-env-dlt2pafg/overlay/lib/python3.11/site-packages/pythran/toolchain.py", line 477, in compile_pythranfile
          output_file = compile_pythrancode(module_name, fd.read(),
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-dlt2pafg/overlay/lib/python3.11/site-packages/pythran/toolchain.py", line 395, in compile_pythrancode
          module, error_checker = generate_cxx(module_name, pythrancode, specs, opts,
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-dlt2pafg/overlay/lib/python3.11/site-packages/pythran/toolchain.py", line 138, in generate_cxx
          pm, ir, docstrings = front_middle_end(module_name, code, optimizations,
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-dlt2pafg/overlay/lib/python3.11/site-packages/pythran/toolchain.py", line 105, in front_middle_end
          refine(pm, ir, optimizations, report_times)
        File "/tmp/pip-build-env-dlt2pafg/overlay/lib/python3.11/site-packages/pythran/middlend.py", line 63, in refine
          apply_optimisation |= pm.apply(optimization, node, run_times)[0]
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-dlt2pafg/overlay/lib/python3.11/site-packages/pythran/passmanager.py", line 243, in apply
          ret=a.apply(node)
              ^^^^^^^^^^^^^
        File "/tmp/pip-build-env-dlt2pafg/overlay/lib/python3.11/site-packages/pythran/passmanager.py", line 201, in apply
          new_node = self.run(node)
                     ^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-dlt2pafg/overlay/lib/python3.11/site-packages/pythran/passmanager.py", line 189, in run
          n = super(Transformation, self).run(node)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-dlt2pafg/overlay/lib/python3.11/site-packages/pythran/passmanager.py", line 97, in run
          return self.visit(node)
                 ^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-dlt2pafg/overlay/lib/python3.11/site-packages/pythran/optimizations/pattern_transform.py", line 327, in visit
          matcher = pattern()
                    ^^^^^^^^^
      TypeError: 'module' object is not callable
      [791/1572] Linking target scipy/stats/_levy_stable/levyst.cpython-311-x86_64-linux-gnu.so
      [792/1572] Compiling C object scipy/stats/_unuran/unuran_wrapper.cpython-311-x86_64-linux-gnu.so.p/.._..__lib_unuran_unuran_src_distr_cxtrans.c.o
      [793/1572] Compiling C object scipy/stats/_unuran/unuran_wrapper.cpython-311-x86_64-linux-gnu.so.p/.._..__lib_unuran_unuran_src_distr_cvec.c.o
      [794/1572] Compiling C++ object scipy/stats/_biasedurn.cpython-311-x86_64-linux-gnu.so.p/meson-generated__biasedurn.cpp.o
      [795/1572] Generating 'scipy/stats/_unuran/unuran_wrapper.cpython-311-x86_64-linux-gnu.so.p/unuran_wrapper.c'.
      [796/1572] Compiling C object scipy/stats/_sobol.cpython-311-x86_64-linux-gnu.so.p/meson-generated__sobol.c.o
      [797/1572] Compiling C++ object scipy/stats/_qmc_cy.cpython-311-x86_64-linux-gnu.so.p/meson-generated__qmc_cy.cpp.o
      [798/1572] Compiling C++ object scipy/stats/_boost/hypergeom_ufunc.cpython-311-x86_64-linux-gnu.so.p/meson-generated_hypergeom_ufunc.cpp.o
      [799/1572] Compiling C++ object scipy/stats/_boost/ncf_ufunc.cpython-311-x86_64-linux-gnu.so.p/meson-generated_ncf_ufunc.cpp.o
      [800/1572] Compiling C++ object scipy/stats/_boost/binom_ufunc.cpython-311-x86_64-linux-gnu.so.p/meson-generated_binom_ufunc.cpp.o
      [801/1572] Compiling C++ object scipy/stats/_boost/nbinom_ufunc.cpython-311-x86_64-linux-gnu.so.p/meson-generated_nbinom_ufunc.cpp.o
      [802/1572] Compiling C++ object scipy/stats/_boost/beta_ufunc.cpython-311-x86_64-linux-gnu.so.p/meson-generated_beta_ufunc.cpp.o
      [803/1572] Compiling C object scipy/stats/_stats.cpython-311-x86_64-linux-gnu.so.p/meson-generated__stats.c.o
      [804/1572] Compiling C object scipy/special/cython_special.cpython-311-x86_64-linux-gnu.so.p/meson-generated_cython_special.c.o
      [805/1572] Compiling C object scipy/linalg/_flapack.cpython-311-x86_64-linux-gnu.so.p/meson-generated_..__flapackmodule.c.o
      [806/1572] Compiling C++ object scipy/sparse/sparsetools/_sparsetools.cpython-311-x86_64-linux-gnu.so.p/csr.cxx.o
      [807/1572] Compiling C++ object scipy/sparse/sparsetools/_sparsetools.cpython-311-x86_64-linux-gnu.so.p/bsr.cxx.o
      ninja: build stopped: subcommand failed.
      + meson setup --native-file=/tmp/pip-install-gc4q_4_2/scipy_f4d6dfa5117d474889b57c3df2985dc6/.mesonpy-native-file.ini -Ddebug=false -Doptimization=2 --prefix=/usr/local /tmp/pip-install-gc4q_4_2/scipy_f4d6dfa5117d474889b57c3df2985dc6 /tmp/pip-install-gc4q_4_2/scipy_f4d6dfa5117d474889b57c3df2985dc6/.mesonpy-a30s00wg/build
      + meson compile
      Traceback (most recent call last):
        File "/home/sparky4/.local/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/home/sparky4/.local/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/home/sparky4/.local/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-dlt2pafg/overlay/lib/python3.11/site-packages/mesonpy/__init__.py", line 970, in get_requires_for_build_wheel
          if not project.is_pure and platform.system() == 'Linux':
                 ^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-dlt2pafg/overlay/lib/python3.11/site-packages/mesonpy/__init__.py", line 875, in is_pure
          return bool(self._wheel_builder.is_pure)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/usr/local/lib/python3.11/functools.py", line 1001, in __get__
          val = self.func(instance)
                ^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-dlt2pafg/overlay/lib/python3.11/site-packages/mesonpy/__init__.py", line 209, in is_pure
          if self._wheel_files['platlib']:
             ^^^^^^^^^^^^^^^^^
        File "/usr/local/lib/python3.11/functools.py", line 1001, in __get__
          val = self.func(instance)
                ^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-dlt2pafg/overlay/lib/python3.11/site-packages/mesonpy/__init__.py", line 170, in _wheel_files
          return self._map_to_wheel(self._sources, self._copy_files)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-dlt2pafg/overlay/lib/python3.11/site-packages/mesonpy/__init__.py", line 512, in _map_to_wheel
          or self._map_from_heuristics(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-dlt2pafg/overlay/lib/python3.11/site-packages/mesonpy/__init__.py", line 476, in _map_from_heuristics
          self._warn_unsure_platlib(origin, destination)
        File "/tmp/pip-build-env-dlt2pafg/overlay/lib/python3.11/site-packages/mesonpy/__init__.py", line 438, in _warn_unsure_platlib
          if self._is_native(origin):
             ^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-dlt2pafg/overlay/lib/python3.11/site-packages/mesonpy/__init__.py", line 405, in _is_native
          self._project.build()  # the project needs to be built for this :/
          ^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-dlt2pafg/overlay/lib/python3.11/site-packages/mesonpy/__init__.py", line 765, in build
          self._meson('compile')
        File "/tmp/pip-build-env-dlt2pafg/overlay/lib/python3.11/site-packages/mesonpy/__init__.py", line 696, in _meson
          return self._proc('meson', *args)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-dlt2pafg/overlay/lib/python3.11/site-packages/mesonpy/__init__.py", line 691, in _proc
          subprocess.check_call(list(args))
        File "/usr/local/lib/python3.11/subprocess.py", line 413, in check_call
          raise CalledProcessError(retcode, cmd)
      subprocess.CalledProcessError: Command '['meson', 'compile']' returned non-zero exit status 1.
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

i need help when i try to run pip -r requierments.txt it throws this error

sparky4 commented 1 year ago

i use Debian 12 and Python 3.11

WASasquatch commented 1 year ago

I think ComfyUI is for Python 3.10, that's what it ships under for the portable. I don't have a Debian distro available to test on but it looks like an issue with scipy.

See this topic regarding your error: https://github.com/scipy/scipy/issues/16784

WASasquatch commented 1 year ago

Maybe try the installation steps provided at the end, may work for your distro as well, I don't see anything specifically "raspberry pi" about the packages.

sparky4 commented 1 year ago

okay~ b

Lightsockie commented 1 year ago

Commenting here instead of a new ticket because I think I'm experiencing the same root issue.

Was getting a similar (but mostly different) error when trying to install scipy via the requirements.txt, and cherrypicking this out of it:

      ../../scipy/meson.build:130:0: ERROR: Dependency "OpenBLAS" not found, tried pkgconfig and cmake

I see it's the same root issue as what the person in that comment WASasquatch mentions fixing on their Pi, from that scipy issue. So installing the package (for Artix, its not blas64-openblas but just blas-openblas I guess), and re-trying the install. Notably I guess these existing packages weren't good enough, lol.

$ sudo pacman -S blas-openblas
resolving dependencies...
looking for conflicting packages...
:: blas-openblas and blas are in conflict. Remove blas? [y/N] y
:: blas-openblas and cblas are in conflict. Remove cblas? [y/N] y
:: blas-openblas and lapack are in conflict. Remove lapack? [y/N] y

(Of note, the installation docs for Scipy seems to think openblas provides "libatlas-base-dev", to get ahead of that potential question I also had.)

Still fails to install via requirements.txt, but now I'm getting the same error as OP.

fwiw I already do have scipy 1.10.1 installed

$ python -m pip freeze | grep sci      
scikit-image==0.20.0
scipy==1.10.1
$ python -m pip install scipy         
Requirement already satisfied: scipy in /home/user/Work/StableDiffusion/ComfyUI/venv/lib/python3.11/site-packages (1.10.1)
Requirement already satisfied: numpy<1.27.0,>=1.19.5 in /home/user/Work/StableDiffusion/ComfyUI/venv/lib/python3.11/site-packages (from scipy) (1.23.5)

So I'm kinda confused what the actual problem is.

Full error log: https://paste.ee/p/psPUt (I think the copy/paste for this got a little wonk, but it's basically all here) pip freeze: https://paste.ee/p/IGihX


I think ComfyUI is for Python 3.10,

fwiw @WASasquatch, I asked comfyanonymous and he says there is no 3.10 target and uses 3.11 himself. I've been using 3.11 for a while with your stuff as well (without ever even knowing there was a 3.10/3.11 issue).

Lightsockie commented 1 year ago

For posterity, I was getting this error in UI when using a CLIPTextEncode (BlenderNeko Advanced + NSP) node. (WAS reported it loaded fine during ComfyUI startup btw, despite me previously ignoring the scipy issue)

WAS Node Suite: CLIPTextEncode Prased Prompt:
 <proompt omit'd>
WAS Node Suite Warning: It doesn't seem ComfyUI_ADV_CLIP_emb is up to date. Falling back to legacy method.
!!! Exception during processing !!!
Traceback (most recent call last):
  File "/home/user/Work/StableDiffusion/ComfyUI/execution.py", line 145, in recursive_execute
    output_data, output_ui = get_output_data(obj, input_data_all)
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/Work/StableDiffusion/ComfyUI/execution.py", line 75, in get_output_data
    return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/Work/StableDiffusion/ComfyUI/execution.py", line 68, in map_node_over_list
    results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/Work/StableDiffusion/ComfyUI/custom_nodes/was-node-suite-comfyui/WAS_Node_Suite.py", line 12692, in encode
    return ([[res[0][0][0], {}]], new_text, text, { "ui": { "string": new_text } } )
              ^^^
NameError: name 'res' is not defined

I'm wildly guessing that the installation stuff is the root issue, but maybe there's another cause. I get this error when using the "Load Default" layout, and replacing one of the CLIP Text Encoders with the WAS version.

I can open a new issue if you think it's unrelated, WASasquatch

(Also, re "WAS Node Suite Warning: It doesn't seem ComfyUI_ADV_CLIP_emb is up to date.", there are no new upstream commits to pull down. I'm on commit e17c01fd3c48060b645f47cdb0c4f63806a997d7 )

WASasquatch commented 1 year ago

Commenting here instead of a new ticket because I think I'm experiencing the same root issue.

Was getting a similar (but mostly different) error when trying to install scipy via the requirements.txt, and cherrypicking this out of it:

      ../../scipy/meson.build:130:0: ERROR: Dependency "OpenBLAS" not found, tried pkgconfig and cmake

I see it's the same root issue as what the person in that comment WASasquatch mentions fixing on their Pi, from that scipy issue. So installing the package (for Artix, its not blas64-openblas but just blas-openblas I guess), and re-trying the install. Notably I guess these existing packages weren't good enough, lol.

$ sudo pacman -S blas-openblas
resolving dependencies...
looking for conflicting packages...
:: blas-openblas and blas are in conflict. Remove blas? [y/N] y
:: blas-openblas and cblas are in conflict. Remove cblas? [y/N] y
:: blas-openblas and lapack are in conflict. Remove lapack? [y/N] y

(Of note, the installation docs for Scipy seems to think openblas provides "libatlas-base-dev", to get ahead of that potential question I also had.)

Still fails to install via requirements.txt, but now I'm getting the same error as OP.

fwiw I already do have scipy 1.10.1 installed

$ python -m pip freeze | grep sci      
scikit-image==0.20.0
scipy==1.10.1
$ python -m pip install scipy         
Requirement already satisfied: scipy in /home/user/Work/StableDiffusion/ComfyUI/venv/lib/python3.11/site-packages (1.10.1)
Requirement already satisfied: numpy<1.27.0,>=1.19.5 in /home/user/Work/StableDiffusion/ComfyUI/venv/lib/python3.11/site-packages (from scipy) (1.23.5)

So I'm kinda confused what the actual problem is.

Full error log: https://paste.ee/p/psPUt (I think the copy/paste for this got a little wonk, but it's basically all here) pip freeze: https://paste.ee/p/IGihX

I think ComfyUI is for Python 3.10,

fwiw @WASasquatch, I asked comfyanonymous and he says there is no 3.10 target and uses 3.11 himself. I've been using 3.11 for a while with your stuff as well (without ever even knowing there was a 3.10/3.11 issue).

ComfyUI rides the latest releases so it itself will work on 3.11, but most the AI scene does not do this because most people use cloud compute and colabs which have old python builds like 3.8 and 3.9. Most these packages don't have 3.11 let alone 3.10 in mind.

With that knowledge you can't expect things using packages for 3.8 to just work on 3.11 without doing the compiling yourself which isn't a WAS-NS or ComfyUI thing, but something you have to take on.

This is why using portable is important. It comes with these tools you would need to compile.

You can ignore the Asv clip warning. Blender added, then removed a feature which I just quickly patched from GitHub.

Le09 commented 1 year ago

With that knowledge you can't expect things using packages for 3.8 to just work on 3.11 without doing the compiling yourself which isn't a WAS-NS or ComfyUI thing, but something you have to take on.

You're obviously not responsible for supporting every environment, especially when the python ecosystem is so brittle. However it would be good to know, in the README's install instructions, that only 3.8 is tested/supported?

Checking the ComfyUI instructions, they do mention to use some python <3.12. It's a bit painful to have to luckily find this thread to create a new env.

Only thing that's tested is Python 3.10, that ComfyUI portable comes with and 90% of used use. That's why say 3.10. Comfyanonymous can use whatever but fact of the matter is this whole time portable, which is mostly used, and where most nodes are created and targeted, is python 3.10. This is why ComfyUI needs to get with the program and target pinned deps and specific python so everyone is on the same page with errors, not all over the place.