search

koaning / scikit-lego

Extra blocks for scikit-learn pipelines.
https://koaning.github.io/scikit-lego/
MIT License
1.29k stars 118 forks source link

Large Volumes of Tests Failing in Local PyTest Run #546

Closed john-hawkins closed 2 years ago

john-hawkins commented 2 years ago

Good morning,

I have forked the project to prepare some modifications to contribute.

I am trying to run the test suite before adding my own tests and I am finding that large numbers of the tests fail on my local machine. I checked the setup.py to confirm that my local machine had the right dependencies and I cannot see anything that should cause a problem.

  1. Is there something I need to do in order to have my local setup run the tests?
  2. Is local testing the correct approach? Or should I just enable the GitHub workflows in the forked repository?

... ERROR tests/test_meta/test_subjective_classifier.py::test_predict_proba[both-expected_probas2] ERROR tests/test_preprocessing/test_outlier_remover.py::test_no_outliers ERROR tests/test_preprocessing/test_outlier_remover.py::test_remove_outlier ERROR tests/test_preprocessing/test_outlier_remover.py::test_do_not_refit ============ 161 failed, 3075 passed, 442 warnings, 17 errors in 107.93s (0:01:47) ============

koaning commented 2 years ago

It would help if you could share some more information. What operating system and Python version are you using?

Technically, it could be that you've installed a recent version of scikit-learn, could you share that version as well?

koaning commented 2 years ago

It does seem like your errors are similar to the ones that failed in our cronjob, indicating this is likely a "new version of sklearn introduced a minor but breaking bug" kind of a thing.

john-hawkins commented 2 years ago

Thanks @koaning

Running on a Mac laptop ProductVersion: 11.3.1 BuildVersion: 20E241

% python --version Python 3.9.15

% pip freeze | grep scikit-learn scikit-learn==1.0.2

koaning commented 2 years ago

I tried reproducing it locally, but didn't find the same issues. Just to check ... are you running on an M1 Mac?

MBrouns commented 2 years ago

Could you post the pytest output to pasteboard or something similar?

john-hawkins commented 2 years ago

@koaning -- It is an Intel Mac. I am going to try with some setup variations.

@MBrouns -- Will do

john-hawkins commented 2 years ago

My apologies everyone.

I have resolved the issue myself.

It was related to not having the all testing packages installed correctly.