search

MaartenGr / BERTopic

Leveraging BERT and c-TF-IDF to create easily interpretable topics.
https://maartengr.github.io/BERTopic/
MIT License
6.05k stars 756 forks source link

Building wheel for hdbscan (pyproject.toml) did not run successfully. #1411

Open Ricksanchez000 opened 1 year ago

Ricksanchez000 commented 1 year ago

WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv class="ansi-yellow-fg"> error: subprocess-exited-with-error

× Building wheel for hdbscan (pyproject.toml) did not run successfully. │ exit code: 1 ╰─> [168 lines of output] running bdist_wheel running build running build_py creating build creating build/lib.linux-x86_64-cpython-310 creating build/lib.linux-x86_64-cpython-310/hdbscan copying hdbscan/validity.py -> build/lib.linux-x86_64-cpython-310/hdbscan copying hdbscan/init.py -> build/lib.linux-x8664-cpython-310/hdbscan copying hdbscan/hdbscan.py -> build/lib.linux-x86_64-cpython-310/hdbscan copying hdbscan/robust_singlelinkage.py -> build/lib.linux-x86_64-cpython-310/hdbscan copying hdbscan/prediction.py -> build/lib.linux-x86_64-cpython-310/hdbscan copying hdbscan/plots.py -> build/lib.linux-x86_64-cpython-310/hdbscan copying hdbscan/flat.py -> build/lib.linux-x86_64-cpython-310/hdbscan creating build/lib.linux-x86_64-cpython-310/hdbscan/tests copying hdbscan/tests/test_prediction_utils.py -> build/lib.linux-x86_64-cpython-310/hdbscan/tests copying hdbscan/tests/init.py -> build/lib.linux-x86_64-cpython-310/hdbscan/tests copying hdbscan/tests/test_hdbscan.py -> build/lib.linux-x86_64-cpython-310/hdbscan/tests copying hdbscan/tests/test_flat.py -> build/lib.linux-x86_64-cpython-310/hdbscan/tests copying hdbscan/tests/test_rsl.py -> build/lib.linux-x86_64-cpython-310/hdbscan/tests running buildext /tmp/pip-build-env-ymbqos4/overlay/lib/python3.10/site-packages/Cython/Compiler/Main.py:381: FutureWarning: Cython directive 'language_level' not set, using '3str' for now (Py3). This has changed from earlier releases! File: /tmp/pip-install-2wkh6f67/hdbscan_d9282b91d1024c7d897cc0e02360a19f/hdbscan/_hdbscan_tree.pyx tree = Parsing.p_module(s, pxd, full_module_name) Compiling hdbscan/_hdbscan_tree.pyx because it changed. [1/1] Cythonizing hdbscan/_hdbscan_tree.pyx building 'hdbscan._hdbscan_tree' extension creating build/temp.linux-x86_64-cpython-310 creating build/temp.linux-x86_64-cpython-310/hdbscan gcc -pthread -B /opt/conda/compilercompat -Wno-unused-result -Wsign-compare -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /opt/conda/include -fPIC -O2 -isystem /opt/conda/include -fPIC -I/opt/conda/include/python3.10 -I/tmp/pip-build-env-ymbqos4/overlay/lib/python3.10/site-packages/numpy/core/include -c hdbscan/_hdbscan_tree.c -o build/temp.linux-x86_64-cpython-310/hdbscan/_hdbscantree.o In file included from /tmp/pip-build-env-ymbqos4/overlay/lib/python3.10/site-packages/numpy/core/include/numpy/ndarraytypes.h:1969, from /tmp/pip-build-env-ymbqos4/overlay/lib/python3.10/site-packages/numpy/core/include/numpy/ndarrayobject.h:12, from /tmp/pip-build-env-ymbqos4/overlay/lib/python3.10/site-packages/numpy/core/include/numpy/arrayobject.h:4, from hdbscan/_hdbscantree.c:1097: /tmp/pip-build-env-ymbqos4/overlay/lib/python3.10/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: #warning "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp] 17 | #warning "Using deprecated NumPy API, disable it with " \ | ^~~ gcc -pthread -B /opt/conda/compiler_compat -shared -Wl,--allow-shlib-undefined -Wl,-rpath,/opt/conda/lib -Wl,-rpath-link,/opt/conda/lib -L/opt/conda/lib -Wl,--allow-shlib-undefined -Wl,-rpath,/opt/conda/lib -Wl,-rpath-link,/opt/conda/lib -L/opt/conda/lib build/temp.linux-x86_64-cpython-310/hdbscan/_hdbscan_tree.o -o build/lib.linux-x86_64-cpython-310/hdbscan/_hdbscan_tree.cpython-310-x8664-linux-gnu.so /tmp/pip-build-env-ymbqos4/overlay/lib/python3.10/site-packages/Cython/Compiler/Main.py:381: FutureWarning: Cython directive 'language_level' not set, using '3str' for now (Py3). This has changed from earlier releases! File: /tmp/pip-install-2wkh6f67/hdbscan_d9282b91d1024c7d897cc0e02360a19f/hdbscan/_hdbscan_linkage.pyx tree = Parsing.p_module(s, pxd, full_module_name)

  Error compiling Cython file:
  ------------------------------------------------------------
  ...
  import numpy as np
  cimport numpy as np

  from libc.float cimport DBL_MAX

  from dist_metrics cimport DistanceMetric
  ^
  ------------------------------------------------------------

  hdbscan/_hdbscan_linkage.pyx:12:0: 'dist_metrics.pxd' not found

  Error compiling Cython file:
  ------------------------------------------------------------
  ...
  import numpy as np
  cimport numpy as np

  from libc.float cimport DBL_MAX

  from dist_metrics cimport DistanceMetric
  ^
  ------------------------------------------------------------

  hdbscan/_hdbscan_linkage.pyx:12:0: 'dist_metrics/DistanceMetric.pxd' not found

  Error compiling Cython file:
  ------------------------------------------------------------
  ...

  cpdef np.ndarray[np.double_t, ndim=2] mst_linkage_core_vector(
          np.ndarray[np.double_t, ndim=2, mode='c'] raw_data,
          np.ndarray[np.double_t, ndim=1, mode='c'] core_distances,
          DistanceMetric dist_metric,
          ^
  ------------------------------------------------------------

  hdbscan/_hdbscan_linkage.pyx:58:8: 'DistanceMetric' is not a type identifier

  Error compiling Cython file:
  ------------------------------------------------------------
  ...
                  continue

              right_value = current_distances[j]
              right_source = current_sources[j]

              left_value = dist_metric.dist(&raw_data_ptr[num_features *
                                            ^
  ------------------------------------------------------------

  hdbscan/_hdbscan_linkage.pyx:129:42: Cannot convert 'double_t *' to Python object

  Error compiling Cython file:
  ------------------------------------------------------------
  ...
              right_value = current_distances[j]
              right_source = current_sources[j]

              left_value = dist_metric.dist(&raw_data_ptr[num_features *
                                                          current_node],
                                            &raw_data_ptr[num_features * j],
                                            ^
  ------------------------------------------------------------

  hdbscan/_hdbscan_linkage.pyx:131:42: Cannot convert 'double_t *' to Python object
  Compiling hdbscan/_hdbscan_linkage.pyx because it changed.
  [1/1] Cythonizing hdbscan/_hdbscan_linkage.pyx
  Traceback (most recent call last):
    File "/opt/conda/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
      main()
    File "/opt/conda/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "/opt/conda/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 251, in build_wheel
      return _build_backend().build_wheel(wheel_directory, config_settings,
    File "/tmp/pip-build-env-ymbqos4_/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 416, in build_wheel
      return self._build_with_temp_dir(['bdist_wheel'], '.whl',
    File "/tmp/pip-build-env-ymbqos4_/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 401, in _build_with_temp_dir
      self.run_setup()
    File "/tmp/pip-build-env-ymbqos4_/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 487, in run_setup
      super(_BuildMetaLegacyBackend,
    File "/tmp/pip-build-env-ymbqos4_/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 338, in run_setup
      exec(code, locals())
    File "<string>", line 96, in <module>
    File "/tmp/pip-build-env-ymbqos4_/overlay/lib/python3.10/site-packages/setuptools/__init__.py", line 107, in setup
      return distutils.core.setup(**attrs)
    File "/tmp/pip-build-env-ymbqos4_/overlay/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 185, in setup
      return run_commands(dist)
    File "/tmp/pip-build-env-ymbqos4_/overlay/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
      dist.run_commands()
    File "/tmp/pip-build-env-ymbqos4_/overlay/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
      self.run_command(cmd)
    File "/tmp/pip-build-env-ymbqos4_/overlay/lib/python3.10/site-packages/setuptools/dist.py", line 1234, in run_command
      super().run_command(command)
    File "/tmp/pip-build-env-ymbqos4_/overlay/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
      cmd_obj.run()
    File "/tmp/pip-build-env-ymbqos4_/overlay/lib/python3.10/site-packages/wheel/bdist_wheel.py", line 343, in run
      self.run_command("build")
    File "/tmp/pip-build-env-ymbqos4_/overlay/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
      self.distribution.run_command(command)
    File "/tmp/pip-build-env-ymbqos4_/overlay/lib/python3.10/site-packages/setuptools/dist.py", line 1234, in run_command
      super().run_command(command)
    File "/tmp/pip-build-env-ymbqos4_/overlay/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
      cmd_obj.run()
    File "/tmp/pip-build-env-ymbqos4_/overlay/lib/python3.10/site-packages/setuptools/_distutils/command/build.py", line 131, in run
      self.run_command(cmd_name)
    File "/tmp/pip-build-env-ymbqos4_/overlay/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
      self.distribution.run_command(command)
    File "/tmp/pip-build-env-ymbqos4_/overlay/lib/python3.10/site-packages/setuptools/dist.py", line 1234, in run_command
      super().run_command(command)
    File "/tmp/pip-build-env-ymbqos4_/overlay/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
      cmd_obj.run()
    File "<string>", line 26, in run
    File "/tmp/pip-build-env-ymbqos4_/overlay/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 345, in run
      self.build_extensions()
    File "/tmp/pip-build-env-ymbqos4_/overlay/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 467, in build_extensions
      self._build_extensions_serial()
    File "/tmp/pip-build-env-ymbqos4_/overlay/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 493, in _build_extensions_serial
      self.build_extension(ext)
    File "/tmp/pip-build-env-ymbqos4_/overlay/lib/python3.10/site-packages/Cython/Distutils/build_ext.py", line 122, in build_extension
      new_ext = cythonize(
    File "/tmp/pip-build-env-ymbqos4_/overlay/lib/python3.10/site-packages/Cython/Build/Dependencies.py", line 1134, in cythonize
      cythonize_one(*args)
    File "/tmp/pip-build-env-ymbqos4_/overlay/lib/python3.10/site-packages/Cython/Build/Dependencies.py", line 1301, in cythonize_one
      raise CompileError(None, pyx_file)
  Cython.Compiler.Errors.CompileError: hdbscan/_hdbscan_linkage.pyx
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for hdbscan ERROR: Could not build wheels for hdbscan, which is required to install pyproject.toml-based projects WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv class="ansi-yellow-fg"> WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv class="ansi-yellow-fg">

Hi, I meet this problem today ,not sure what happened, I can run the same code yesterday, and today just keep meeting this problem with hdbscan , can you please help me with it?

MaartenGr commented 1 year ago

I believe this is [a known issue with HDBSCAN for which a new version was released that hopefully resolved the issue. I also just checked Pypi and saw that a minute ago a new version of HDBSCAN was released. That might just solve the issue.

jburdo1 commented 1 year ago

hdbscan==0.8.31 does indeed fix the problem.

Ricksanchez000 commented 1 year ago

I believe this is [a known issue with HDBSCAN for which a new version was released that hopefully resolved the issue. I also just checked Pypi and saw that a minute ago a new version of HDBSCAN was released. That might just solve the issue.

yeah, thank you so much, I just updated my scikit-learn along with HDBSCAN, and it works again, so happy because it's one of the important topic modeling models I used for my dissertation :)

MaartenGr commented 1 year ago

Glad to hear that it works again. If you run into any other issues, feel free to reach out!

manjunath-hanmantgad commented 1 year ago

Mine got fixed at 0.8.32 v.

MouzaMrinal commented 3 weeks ago

Mine got fixed by using HDBSCAN 0.8.33, earlier it was 0.8.29