search

scikit-learn / scikit-learn

scikit-learn: machine learning in Python
https://scikit-learn.org
BSD 3-Clause "New" or "Revised" License
59.66k stars 25.31k forks source link

_hist_gradient_boosting tests not finishing in pypy CI #13763

Closed jnothman closed 5 years ago

jnothman commented 5 years ago

I don't think we have an issue for this elsewhere... Our pypy builds are failing after no output for 10 minutes when running the new gradient boosting tests.

https://circleci.com/gh/scikit-learn/scikit-learn/56726

thomasjpfan commented 5 years ago

@NicolasHug pypy fun

NicolasHug commented 5 years ago

Can anyone reproduce the timeout?

I just setup a virtualenv with pypy and numpy 1.15 on my machine, and the tests pass correctly :/

rth commented 5 years ago

@NicolasHug You could try running it in the same docker container as CircleCI.

ogrisel commented 5 years ago

This might be caused by an oversubscription issue on circle ci hosts. I remember we had to manually set LOKY_MAX_CPU_COUNT=2 to avoid a similar issue on joblib related tests on that host because pypy does not report the correct number of available CPUs on circle CI for some reason I do not recall exactly.

Maybe setting export OMP_NUM_THREADS=2 in the pypy test script can help mitigate this issue.

jnothman commented 5 years ago

I am getting stimulus fatigue from the "pypy still failing" emails. Let's disable histgb tests on pypy and leave this issue open?

rth commented 5 years ago

I am getting stimulus fatigue from the "pypy still failing" emails.

Unsubscribing from those is the first thing I do.

Anyway, this should be fixed by https://github.com/scikit-learn/scikit-learn/pull/13912. Apparently upgrading to PyPy 7.1.1 makes this issue go away.