search

dask / dask-xgboost

BSD 3-Clause "New" or "Revised" License
162 stars 43 forks source link

[BUG] testcase faliure - gen_cluster() got an unexpected keyword argument 'check_new_threads' #49

Closed ksangeek closed 4 years ago

ksangeek commented 5 years ago

Describe the bug pytest for the tests having call to gen_cluster() fails with -

E   TypeError: gen_cluster() got an unexpected keyword argument 'check_new_threads'

Steps/Code to reproduce bug This can be easily reproduced with the distributed version 2.3.2. Here is the complete error seen on the terminal -

$ pytest test_core.py
====================================== test session starts =======================================
platform linux -- Python 3.6.9, pytest-5.0.1, py-1.8.0, pluggy-0.12.0
rootdir: /home/sangeek/dask-xgb-tests/tests
plugins: cov-2.7.1, xdist-1.28.0, forked-1.0.2
collected 0 items / 1 errors

============================================= ERRORS =============================================
_________________________________ ERROR collecting test_core.py __________________________________
test_core.py:136: in <module>
    @gen_cluster(client=True, timeout=None, check_new_threads=False)
E   TypeError: gen_cluster() got an unexpected keyword argument 'check_new_threads'
!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!
==================================== 1 error in 3.70 seconds =====================================

Additional context Looking at the code for the latest release of distributed - https://github.com/dask/distributed/blob/a0d68d42b81fe0215098d7f87783e2d57b2aa703/distributed/utils_test.py#L831 I see that they have removed support for the parameter check_new_threads.

I see that removing that parameter from the call to gen_cluster(), helps me get past this issue i.e.

< @gen_cluster(client=True, timeout=None, check_new_threads=False)
---
> @gen_cluster(client=True, timeout=None)

I could have attempted to patch this change but for the reasons mentioned in https://github.com/dask/dask-xgboost/issues/47 I can't still run all the tests successfully, so not sure if there are any other side-effects of this change?

mrocklin commented 5 years ago

This should either be removed (if that passes) or replaced with clean_kwargs={"threads": False}