neurodata / scikit-learn

scikit-learn-tree fork: A fork that enables extensions of Python and Cython API for decision trees
https://scikit-learn.org
BSD 3-Clause "New" or "Revised" License
7 stars 6 forks source link

WIP correct CI errors relating to partial_fit #58

Closed PSSF23 closed 1 year ago

PSSF23 commented 1 year ago

Reference Issues/PRs

Fixes #57

What does this implement/fix? Explain your changes.

Any other comments?

github-actions[bot] commented 1 year ago

✔️ Linting Passed

All linting checks passed. Your pull request is in excellent shape! ☀️

Generated for commit: b22d33a. Link to the linter CI: here

adam2392 commented 1 year ago

For the segfault that occurs with honest forest, checkout code here https://github.com/neurodata/scikit-tree/pull/118

and run: pytest ./sktree/tests/test_honest_forest.py::test_sklearn_compatible_estimator

PSSF23 commented 1 year ago

I could not run the test because it always give this import error or check_build, even when sktree (installed through main) runs properly in notebooks and scripts.

ImportError while importing test module '/Users/pssf23/GitHub/scikit-tree/sktree/tests/test_honest_forest.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
scikit-tree/sktree/_lib/sklearn/__check_build/__init__.py:45: in <module>
    from ._check_build import check_build  # noqa
E   ModuleNotFoundError: No module named 'sktree._lib.sklearn.__check_build._check_build'
adam2392 commented 1 year ago

Oh maybe after building using meson. Run the pip install editable line on the readme?

Otw you can't run pytest directly.

adam2392 commented 1 year ago

@adam2392 I have modified the initial_roots and how it's stored in tree builders. Can you test whether Honest Forest work now on your end? I'm so confused by this bug. It seems that I cannot satisfy all system configurations at once (before it was mac, right now it's windows), but at least there should be improvements.

It seems there is still a segfault in https://github.com/neurodata/scikit-tree/pull/118. If you want to try it out:

./spin build -j 6 --clean
pip install --no-build-isolation --editable .
pytest ./sktree/tests/test_honest_forest.py::test_sklearn_compatible_estimator
adam2392 commented 1 year ago

I get:

(sktree) adam2392@adams-mbp-3 scikit-tree % pytest ./sktree/tests/test_honest_forest.py::test_sklearn_compatible_estimator
=============================================== test session starts ================================================
platform darwin -- Python 3.9.15, pytest-7.4.0, pluggy-1.0.0 -- /Users/adam2392/miniforge3/envs/sktree/bin/python3.9
cachedir: .pytest_cache
rootdir: /Users/adam2392/Documents/scikit-tree
configfile: pyproject.toml
plugins: parallel-0.1.1, cov-4.1.0, typeguard-2.13.3
collected 56 items                                                                                                 

sktree/tests/test_honest_forest.py::test_sklearn_compatible_estimator[HonestForestClassifier(n_estimators=10,random_state=0)-check_no_attributes_set_in_init] PASSED [  1%]
sktree/tests/test_honest_forest.py::test_sklearn_compatible_estimator[HonestForestClassifier(n_estimators=10,random_state=0)-check_estimators_dtypes] PASSED [  3%]
sktree/tests/test_honest_forest.py::test_sklearn_compatible_estimator[HonestForestClassifier(n_estimators=10,random_state=0)-check_fit_score_takes_y] Fatal Python error: Segmentation fault

Current thread 0x00000001f8815e00 (most recent call first):
  File "/Users/adam2392/Documents/scikit-tree/sktree/_lib/./sklearn/tree/_classes.py", line 1377 in partial_fit
  File "/Users/adam2392/Documents/scikit-tree/sktree/tree/_honest_tree.py", line 426 in partial_fit
  File "/Users/adam2392/Documents/scikit-tree/sktree/_lib/./sklearn/ensemble/_forest.py", line 252 in _parallel_update_trees
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/sklearn/utils/parallel.py", line 127 in __call__
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/joblib/parallel.py", line 1792 in _get_sequential_output
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/joblib/parallel.py", line 1863 in __call__
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/sklearn/utils/parallel.py", line 65 in __call__
  File "/Users/adam2392/Documents/scikit-tree/sktree/_lib/./sklearn/ensemble/_forest.py", line 1332 in partial_fit
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/sklearn/utils/estimator_checks.py", line 1816 in check_fit_score_takes_y
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/sklearn/utils/_testing.py", line 180 in wrapper
  File "/Users/adam2392/Documents/scikit-tree/sktree/tests/test_honest_forest.py", line 192 in test_sklearn_compatible_estimator
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/_pytest/python.py", line 194 in pytest_pyfunc_call
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/pluggy/_callers.py", line 39 in _multicall
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/pluggy/_manager.py", line 80 in _hookexec
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/pluggy/_hooks.py", line 265 in __call__
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/_pytest/python.py", line 1788 in runtest
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/_pytest/runner.py", line 169 in pytest_runtest_call
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/pluggy/_callers.py", line 39 in _multicall
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/pluggy/_manager.py", line 80 in _hookexec
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/pluggy/_hooks.py", line 265 in __call__
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/_pytest/runner.py", line 262 in <lambda>
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/_pytest/runner.py", line 341 in from_call
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/_pytest/runner.py", line 261 in call_runtest_hook
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/_pytest/runner.py", line 222 in call_and_report
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/_pytest/runner.py", line 133 in runtestprotocol
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/_pytest/runner.py", line 114 in pytest_runtest_protocol
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/pluggy/_callers.py", line 39 in _multicall
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/pluggy/_manager.py", line 80 in _hookexec
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/pluggy/_hooks.py", line 265 in __call__
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/_pytest/main.py", line 349 in pytest_runtestloop
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/pluggy/_callers.py", line 39 in _multicall
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/pluggy/_manager.py", line 80 in _hookexec
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/pluggy/_hooks.py", line 265 in __call__
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/_pytest/main.py", line 324 in _main
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/_pytest/main.py", line 270 in wrap_session
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/_pytest/main.py", line 317 in pytest_cmdline_main
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/pluggy/_callers.py", line 39 in _multicall
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/pluggy/_manager.py", line 80 in _hookexec
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/pluggy/_hooks.py", line 265 in __call__
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/_pytest/config/__init__.py", line 166 in main
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/_pytest/config/__init__.py", line 189 in console_main
  File "/Users/adam2392/miniforge3/envs/sktree/bin/pytest", line 8 in <module>
zsh: segmentation fault  pytest ./sktree/tests/test_honest_forest.py::test_sklearn_compatible_estimato
adam2392 commented 1 year ago

Okay merging in since this seems to fix the CIs. Thanks @PSSF23 !