search

david-cortes / isotree

(Python, R, C/C++) Isolation Forest and variations such as SCiForest and EIF, with some additions (outlier detection + similarity + NA imputation)
https://isotree.readthedocs.io
BSD 2-Clause "Simplified" License
186 stars 38 forks source link

Running isotree in databricks environment given requirements #61

Closed eza494 closed 3 months ago

eza494 commented 3 months ago

Hi I seem to be getting errors trying to install isotree. I have checked with the requirements of the package and I have ensured they are satisfied. Please could you have a look at as well. I would like to apologise as this is similar to another question but I dont know what else to do as I have satisfied the requirements The command is !pip install isotree

Installing build dependencies ... done Getting requirements to build wheel ... done Preparing metadata (pyproject.toml) ... done Building wheels for collected packages: isotree Building wheel for isotree (pyproject.toml) ... error error: subprocess-exited-with-error

× Building wheel for isotree (pyproject.toml) did not run successfully. │ exit code: 1 ╰─> [604 lines of output] /databricks/python/lib/python3.10/site-packages/setuptools/_distutils/extension.py:134: UserWarning: Unknown Extension options: 'install_requires' warnings.warn(msg) /databricks/python/lib/python3.10/site-packages/setuptools/dist.py:530: UserWarning: Normalizing '0.6.1-2' to '0.6.1.post2' warnings.warn(tmpl.format(**locals())) running bdist_wheel running build running build_py creating build creating build/lib.linux-x86_64-cpython-310 creating build/lib.linux-x86_64-cpython-310/isotree copying isotree/init.py -> build/lib.linux-x86_64-cpython-310/isotree running build_ext --- Checking compiler support for option '-march=native' --- Checking compiler support for option '-fopenmp' --- Checking compiler support for '__restrict' qualifier --- Checking compiler support for option '-O3' --- Checking compiler support for option '-fno-math-errno' --- Checking compiler support for option '-fno-trapping-math' --- Checking compiler support for option '-std=c++17' --- Checking compiler support for option '-flto=auto' cythoning isotree/cpp_interface.pyx to isotree/cpp_interface.cpp

  Error compiling Cython file:
  ------------------------------------------------------------
  ...
      void tmat_to_dense(double *tmat, double *dmat, size_t n, double fill_diag)

      void merge_models(IsoForest*     model,      const IsoForest*     other,
                        ExtIsoForest*  ext_model,  const ExtIsoForest*  ext_other,
                        Imputer*       imputer,    const Imputer*       iother,
                        TreesIndexer*  indexer,    const TreesIndexer*  ind_other) except + nogil
                                                                                                ^
  ------------------------------------------------------------

  isotree/cpp_interface.pyx:223:95: undeclared name not builtin: nogil

  Error compiling Cython file:
  ------------------------------------------------------------
  ...

          cdef int ret_val = 0

          with nogil, boundscheck(False), nonecheck(False), wraparound(False):
              ret_val = \
              fit_iforest(model_ptr, ext_model_ptr,
                        ^
  ------------------------------------------------------------

  isotree/cpp_interface.pyx:841:23: Calling gil-requiring function not allowed without gil

  Error compiling Cython file:
  ------------------------------------------------------------
  ...

          cdef int ret_val = 0

          with nogil, boundscheck(False), nonecheck(False), wraparound(False):
              ret_val = \
              fit_iforest(model_ptr, ext_model_ptr,
                        ^
  ------------------------------------------------------------

  isotree/cpp_interface.pyx:841:23: Calling gil-requiring function not allowed without gil

  Error compiling Cython file:
  ------------------------------------------------------------
  ...

          cdef int ret_val = 0

          with nogil, boundscheck(False), nonecheck(False), wraparound(False):
              ret_val = \
              fit_iforest(model_ptr, ext_model_ptr,
                        ^
  ------------------------------------------------------------

  isotree/cpp_interface.pyx:841:23: Calling gil-requiring function not allowed without gil

  Error compiling Cython file:
  ------------------------------------------------------------
  ...

          cdef int ret_val = 0

          with nogil, boundscheck(False), nonecheck(False), wraparound(False):
              ret_val = \
              fit_iforest(model_ptr, ext_model_ptr,
                        ^
  ------------------------------------------------------------

  isotree/cpp_interface.pyx:841:23: Calling gil-requiring function not allowed without gil

  Error compiling Cython file:
  ------------------------------------------------------------
  ...

          cdef int ret_val = 0

          with nogil, boundscheck(False), nonecheck(False), wraparound(False):
              ret_val = \
              fit_iforest(model_ptr, ext_model_ptr,
                        ^
  ------------------------------------------------------------

  isotree/cpp_interface.pyx:841:23: Calling gil-requiring function not allowed without gil

  Error compiling Cython file:
  ------------------------------------------------------------
  ...

          cdef int ret_val = 0

          with nogil, boundscheck(False), nonecheck(False), wraparound(False):
              ret_val = \
              fit_iforest(model_ptr, ext_model_ptr,
                        ^
  ------------------------------------------------------------

  isotree/cpp_interface.pyx:841:23: Calling gil-requiring function not allowed without gil

  Error compiling Cython file:
  ------------------------------------------------------------
  ...
          cdef TreesIndexer *indexer_ptr = NULL
          if not self.indexer.indices.empty():
              indexer_ptr = &self.indexer

          with nogil, boundscheck(False), nonecheck(False), wraparound(False):
              add_tree(model_ptr, ext_model_ptr,
                     ^
  ------------------------------------------------------------

  isotree/cpp_interface.pyx:1042:20: Calling gil-requiring function not allowed without gil

  Error compiling Cython file:
  ------------------------------------------------------------
  ...
          cdef TreesIndexer *indexer_ptr = NULL
          if not self.indexer.indices.empty():
              indexer_ptr = &self.indexer

          with nogil, boundscheck(False), nonecheck(False), wraparound(False):
              add_tree(model_ptr, ext_model_ptr,
                     ^
  ------------------------------------------------------------

  isotree/cpp_interface.pyx:1042:20: Calling gil-requiring function not allowed without gil

  Error compiling Cython file:
  ------------------------------------------------------------
  ...
          cdef TreesIndexer *indexer_ptr = NULL
          if not self.indexer.indices.empty():
              indexer_ptr = &self.indexer

          with nogil, boundscheck(False), nonecheck(False), wraparound(False):
              add_tree(model_ptr, ext_model_ptr,
                     ^
  ------------------------------------------------------------

  isotree/cpp_interface.pyx:1042:20: Calling gil-requiring function not allowed without gil

  Error compiling Cython file:
  ------------------------------------------------------------
  ...
          cdef TreesIndexer *indexer_ptr = NULL
          if not self.indexer.indices.empty():
              indexer_ptr = &self.indexer

          with nogil, boundscheck(False), nonecheck(False), wraparound(False):
              add_tree(model_ptr, ext_model_ptr,
                     ^
  ------------------------------------------------------------

  isotree/cpp_interface.pyx:1042:20: Calling gil-requiring function not allowed without gil

  Error compiling Cython file:
  ------------------------------------------------------------
  ...
          cdef TreesIndexer *indexer_ptr = NULL
          if not self.indexer.indices.empty():
              indexer_ptr = &self.indexer

          with nogil, boundscheck(False), nonecheck(False), wraparound(False):
              add_tree(model_ptr, ext_model_ptr,
                     ^
  ------------------------------------------------------------

  isotree/cpp_interface.pyx:1042:20: Calling gil-requiring function not allowed without gil

  Error compiling Cython file:
  ------------------------------------------------------------
  ...
          cdef TreesIndexer *indexer_ptr = NULL
          if not self.indexer.indices.empty():
              indexer_ptr = &self.indexer

          with nogil, boundscheck(False), nonecheck(False), wraparound(False):
              add_tree(model_ptr, ext_model_ptr,
                     ^
  ------------------------------------------------------------

  isotree/cpp_interface.pyx:1042:20: Calling gil-requiring function not allowed without gil

  Error compiling Cython file:
  ------------------------------------------------------------
  ...
          cdef TreesIndexer*  indexer_ptr = NULL
          if not self.indexer.indices.empty():
              indexer_ptr = &self.indexer

          with nogil, boundscheck(False), nonecheck(False), wraparound(False):
              predict_iforest(numeric_data_ptr, categ_data_ptr,
                            ^
  ------------------------------------------------------------

  isotree/cpp_interface.pyx:1181:27: Calling gil-requiring function not allowed without gil

  Error compiling Cython file:
  ------------------------------------------------------------
  ...
          cdef TreesIndexer*  indexer_ptr = NULL
          if not self.indexer.indices.empty():
              indexer_ptr = &self.indexer

          with nogil, boundscheck(False), nonecheck(False), wraparound(False):
              predict_iforest(numeric_data_ptr, categ_data_ptr,
                            ^
  ------------------------------------------------------------

  isotree/cpp_interface.pyx:1181:27: Calling gil-requiring function not allowed without gil

  Error compiling Cython file:
  ------------------------------------------------------------
  ...
          cdef TreesIndexer*  indexer_ptr = NULL
          if not self.indexer.indices.empty():
              indexer_ptr = &self.indexer

          with nogil, boundscheck(False), nonecheck(False), wraparound(False):
              predict_iforest(numeric_data_ptr, categ_data_ptr,
                            ^
  ------------------------------------------------------------

  isotree/cpp_interface.pyx:1181:27: Calling gil-requiring function not allowed without gil

  Error compiling Cython file:
  ------------------------------------------------------------
  ...
          cdef TreesIndexer*  indexer_ptr = NULL
          if not self.indexer.indices.empty():
              indexer_ptr = &self.indexer

          with nogil, boundscheck(False), nonecheck(False), wraparound(False):
              predict_iforest(numeric_data_ptr, categ_data_ptr,
                            ^
  ------------------------------------------------------------

  isotree/cpp_interface.pyx:1181:27: Calling gil-requiring function not allowed without gil

  Error compiling Cython file:
  ------------------------------------------------------------
  ...
          cdef TreesIndexer*  indexer_ptr = NULL
          if not self.indexer.indices.empty():
              indexer_ptr = &self.indexer

          with nogil, boundscheck(False), nonecheck(False), wraparound(False):
              predict_iforest(numeric_data_ptr, categ_data_ptr,
                            ^
  ------------------------------------------------------------

  isotree/cpp_interface.pyx:1181:27: Calling gil-requiring function not allowed without gil

  Error compiling Cython file:
  ------------------------------------------------------------
  ...
          cdef TreesIndexer*  indexer_ptr = NULL
          if not self.indexer.indices.empty():
              indexer_ptr = &self.indexer

          with nogil, boundscheck(False), nonecheck(False), wraparound(False):
              predict_iforest(numeric_data_ptr, categ_data_ptr,
                            ^
  ------------------------------------------------------------

  isotree/cpp_interface.pyx:1181:27: Calling gil-requiring function not allowed without gil

  Error compiling Cython file:
  ------------------------------------------------------------
  ...
          cdef TreesIndexer*  indexer_ptr = NULL
          if not self.indexer.indices.empty():
              indexer_ptr = &self.indexer

          with nogil, boundscheck(False), nonecheck(False), wraparound(False):
              calc_similarity(numeric_data_ptr, categ_data_ptr,
                            ^
  ------------------------------------------------------------

  isotree/cpp_interface.pyx:1305:27: Calling gil-requiring function not allowed without gil

  Error compiling Cython file:
  ------------------------------------------------------------
  ...
          cdef TreesIndexer*  indexer_ptr = NULL
          if not self.indexer.indices.empty():
              indexer_ptr = &self.indexer

          with nogil, boundscheck(False), nonecheck(False), wraparound(False):
              calc_similarity(numeric_data_ptr, categ_data_ptr,
                            ^
  ------------------------------------------------------------

  isotree/cpp_interface.pyx:1305:27: Calling gil-requiring function not allowed without gil

  Error compiling Cython file:
  ------------------------------------------------------------
  ...
          cdef TreesIndexer*  indexer_ptr = NULL
          if not self.indexer.indices.empty():
              indexer_ptr = &self.indexer

          with nogil, boundscheck(False), nonecheck(False), wraparound(False):
              calc_similarity(numeric_data_ptr, categ_data_ptr,
                            ^
  ------------------------------------------------------------

  isotree/cpp_interface.pyx:1305:27: Calling gil-requiring function not allowed without gil

  Error compiling Cython file:
  ------------------------------------------------------------
  ...
          cdef TreesIndexer*  indexer_ptr = NULL
          if not self.indexer.indices.empty():
              indexer_ptr = &self.indexer

          with nogil, boundscheck(False), nonecheck(False), wraparound(False):
              calc_similarity(numeric_data_ptr, categ_data_ptr,
                            ^
  ------------------------------------------------------------

  isotree/cpp_interface.pyx:1305:27: Calling gil-requiring function not allowed without gil

  Error compiling Cython file:
  ------------------------------------------------------------
  ...
          cdef TreesIndexer*  indexer_ptr = NULL
          if not self.indexer.indices.empty():
              indexer_ptr = &self.indexer

          with nogil, boundscheck(False), nonecheck(False), wraparound(False):
              calc_similarity(numeric_data_ptr, categ_data_ptr,
                            ^
  ------------------------------------------------------------

  isotree/cpp_interface.pyx:1305:27: Calling gil-requiring function not allowed without gil

  Error compiling Cython file:
  ------------------------------------------------------------
  ...
          cdef TreesIndexer*  indexer_ptr = NULL
          if not self.indexer.indices.empty():
              indexer_ptr = &self.indexer

          with nogil, boundscheck(False), nonecheck(False), wraparound(False):
              calc_similarity(numeric_data_ptr, categ_data_ptr,
                            ^
  ------------------------------------------------------------

  isotree/cpp_interface.pyx:1305:27: Calling gil-requiring function not allowed without gil

  Error compiling Cython file:
  ------------------------------------------------------------
  ...
              model_ptr      =  &self.isoforest
          else:
              ext_model_ptr  =  &self.ext_isoforest

          with nogil, boundscheck(False), nonecheck(False), wraparound(False):
              impute_missing_values(numeric_data_ptr, categ_data_ptr, is_col_major,
                                  ^
  ------------------------------------------------------------

  isotree/cpp_interface.pyx:1391:33: Calling gil-requiring function not allowed without gil

  Error compiling Cython file:
  ------------------------------------------------------------
  ...
              model_ptr      =  &self.isoforest
          else:
              ext_model_ptr  =  &self.ext_isoforest

          with nogil, boundscheck(False), nonecheck(False), wraparound(False):
              impute_missing_values(numeric_data_ptr, categ_data_ptr, is_col_major,
                                  ^
  ------------------------------------------------------------

  isotree/cpp_interface.pyx:1391:33: Calling gil-requiring function not allowed without gil

  Error compiling Cython file:
  ------------------------------------------------------------
  ...
              model_ptr      =  &self.isoforest
          else:
              ext_model_ptr  =  &self.ext_isoforest

          with nogil, boundscheck(False), nonecheck(False), wraparound(False):
              impute_missing_values(numeric_data_ptr, categ_data_ptr, is_col_major,
                                  ^
  ------------------------------------------------------------

  isotree/cpp_interface.pyx:1391:33: Calling gil-requiring function not allowed without gil

  Error compiling Cython file:
  ------------------------------------------------------------
  ...
              model_ptr      =  &self.isoforest
          else:
              ext_model_ptr  =  &self.ext_isoforest

          with nogil, boundscheck(False), nonecheck(False), wraparound(False):
              impute_missing_values(numeric_data_ptr, categ_data_ptr, is_col_major,
                                  ^
  ------------------------------------------------------------

  isotree/cpp_interface.pyx:1391:33: Calling gil-requiring function not allowed without gil

  Error compiling Cython file:
  ------------------------------------------------------------
  ...
              model_ptr      =  &self.isoforest
          else:
              ext_model_ptr  =  &self.ext_isoforest

          with nogil, boundscheck(False), nonecheck(False), wraparound(False):
              impute_missing_values(numeric_data_ptr, categ_data_ptr, is_col_major,
                                  ^
  ------------------------------------------------------------

  isotree/cpp_interface.pyx:1391:33: Calling gil-requiring function not allowed without gil

  Error compiling Cython file:
  ------------------------------------------------------------
  ...
              model_ptr      =  &self.isoforest
          else:
              ext_model_ptr  =  &self.ext_isoforest

          with nogil, boundscheck(False), nonecheck(False), wraparound(False):
              impute_missing_values(numeric_data_ptr, categ_data_ptr, is_col_major,
                                  ^
  ------------------------------------------------------------

  isotree/cpp_interface.pyx:1391:33: Calling gil-requiring function not allowed without gil

  Error compiling Cython file:
  ------------------------------------------------------------
  ...
              ptr_indexer = &self.indexer
          if not other.indexer.indices.empty():
              prt_ind_other = &other.indexer

          with nogil, boundscheck(False), nonecheck(False), wraparound(False):
              merge_models(ptr_model, ptr_other,
                         ^
  ------------------------------------------------------------

  isotree/cpp_interface.pyx:1441:24: Calling gil-requiring function not allowed without gil

  Error compiling Cython file:
  ------------------------------------------------------------
  ...
          else:
              ext_model_ptr  =  &self.ext_isoforest

          cdef vector[cpp_string] res
          with nogil, boundscheck(False), nonecheck(False), wraparound(False):
              res = generate_sql(model_ptr, ext_model_ptr,
                               ^
  ------------------------------------------------------------

  isotree/cpp_interface.pyx:1563:30: Calling gil-requiring function not allowed without gil

  Error compiling Cython file:
  ------------------------------------------------------------
  ...
          if not self.indexer.indices.empty():
              indexer = &self.indexer

          cdef vector[cpp_string] res
          with nogil, boundscheck(False), nonecheck(False), wraparound(False):
              res = generate_dot(model_ptr, ext_model_ptr, indexer,
                               ^
  ------------------------------------------------------------

  isotree/cpp_interface.pyx:1587:30: Calling gil-requiring function not allowed without gil

  Error compiling Cython file:
  ------------------------------------------------------------
  ...
          if not self.indexer.indices.empty():
              indexer = &self.indexer

          cdef vector[cpp_string] res
          with nogil, boundscheck(False), nonecheck(False), wraparound(False):
              res = generate_json(model_ptr, ext_model_ptr, indexer,
                                ^
  ------------------------------------------------------------

  isotree/cpp_interface.pyx:1611:31: Calling gil-requiring function not allowed without gil

  Error compiling Cython file:
  ------------------------------------------------------------
  ...
          else:
              ext_model_ptr  =  &self.ext_isoforest
          cdef TreesIndexer*  indexer_ptr = &self.indexer

          with nogil, boundscheck(False), nonecheck(False), wraparound(False):
              set_reference_points(model_ptr, ext_model_ptr, indexer_ptr,
                                 ^
  ------------------------------------------------------------

  isotree/cpp_interface.pyx:1752:32: Calling gil-requiring function not allowed without gil

  Error compiling Cython file:
  ------------------------------------------------------------
  ...
          else:
              ext_model_ptr  =  &self.ext_isoforest
          cdef TreesIndexer*  indexer_ptr = &self.indexer

          with nogil, boundscheck(False), nonecheck(False), wraparound(False):
              set_reference_points(model_ptr, ext_model_ptr, indexer_ptr,
                                 ^
  ------------------------------------------------------------

  isotree/cpp_interface.pyx:1752:32: Calling gil-requiring function not allowed without gil

  Error compiling Cython file:
  ------------------------------------------------------------
  ...
          else:
              ext_model_ptr  =  &self.ext_isoforest
          cdef TreesIndexer*  indexer_ptr = &self.indexer

          with nogil, boundscheck(False), nonecheck(False), wraparound(False):
              set_reference_points(model_ptr, ext_model_ptr, indexer_ptr,
                                 ^
  ------------------------------------------------------------

  isotree/cpp_interface.pyx:1752:32: Calling gil-requiring function not allowed without gil

  Error compiling Cython file:
  ------------------------------------------------------------
  ...
          else:
              ext_model_ptr  =  &self.ext_isoforest
          cdef TreesIndexer*  indexer_ptr = &self.indexer

          with nogil, boundscheck(False), nonecheck(False), wraparound(False):
              set_reference_points(model_ptr, ext_model_ptr, indexer_ptr,
                                 ^
  ------------------------------------------------------------

  isotree/cpp_interface.pyx:1752:32: Calling gil-requiring function not allowed without gil

  Error compiling Cython file:
  ------------------------------------------------------------
  ...
          else:
              ext_model_ptr  =  &self.ext_isoforest
          cdef TreesIndexer*  indexer_ptr = &self.indexer

          with nogil, boundscheck(False), nonecheck(False), wraparound(False):
              set_reference_points(model_ptr, ext_model_ptr, indexer_ptr,
                                 ^
  ------------------------------------------------------------

  isotree/cpp_interface.pyx:1752:32: Calling gil-requiring function not allowed without gil

  Error compiling Cython file:
  ------------------------------------------------------------
  ...
          else:
              ext_model_ptr  =  &self.ext_isoforest
          cdef TreesIndexer*  indexer_ptr = &self.indexer

          with nogil, boundscheck(False), nonecheck(False), wraparound(False):
              set_reference_points(model_ptr, ext_model_ptr, indexer_ptr,
                                 ^
  ------------------------------------------------------------

  isotree/cpp_interface.pyx:1752:32: Calling gil-requiring function not allowed without gil
  building 'isotree._cpp_interface' extension
  creating build/temp.linux-x86_64-cpython-310
  creating build/temp.linux-x86_64-cpython-310/isotree
  creating build/temp.linux-x86_64-cpython-310/src
  x86_64-linux-gnu-gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -D_USE_XOSHIRO -DNDEBUG -D_FOR_PYTHON "-DCYTHON_EXTERN_C=extern \"C\"" -DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION -D_USE_ROBIN_MAP -I/databricks/python/lib/python3.10/site-packages/numpy/core/include -I. -I./src -I/local_disk0/.ephemeral_nfs/envs/pythonEnv-20cb01b3-10e5-4c84-aba1-67507ea71074/include -I/usr/include/python3.10 -c isotree/cpp_interface.cpp -o build/temp.linux-x86_64-cpython-310/isotree/cpp_interface.o -march=native -fopenmp -O3 -fno-math-errno -fno-trapping-math -std=c++17 -flto=auto
  isotree/cpp_interface.cpp:1:2: error: #error Do not use this file, it is the result of a failed Cython compilation.
      1 | #error Do not use this file, it is the result of a failed Cython compilation.
        |  ^~~~~
  error: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for isotree Failed to build isotree ERROR: Could not build wheels for isotree, which is required to install pyproject.toml-based projects

david-cortes commented 3 months ago

Looks like you might be using an old cython version. You could try contacting databricks support if updating cython doesn't help.