search

getlogbook / logbook

A cool logging replacement for Python.
http://logbook.readthedocs.org
Other
1.48k stars 164 forks source link

Conflict with the joblib in sklearn #244

Open nonamestreet opened 7 years ago

nonamestreet commented 7 years ago 
import logbook                                                                  
from sklearn.cluster import KMeans                                              
import numpy as np                                                              

if __name__ == '__main__':                                                      
    data = np.random.rand(1000, 10)                                             
    kmeans = KMeans(n_clusters=10, n_jobs=2)                                    
    kmeans.fit(data)

Output warning:

/home/sunlin/anaconda2/lib/python2.7/site-packages/sklearn/externals/joblib/parallel.py:547: UserWarning: Multiprocessing-backed parallel loops cannot be nested below threads, setting n_jobs=1 **self._backend_args)

This essentially disabled the parallel processing. The code works fine when the import logbook is commented out.

Logbook version: 1.1.0 sklearn version: 0.19.0 Python version: Python 2.7.13

vmalloc commented 7 years ago

@nonamestreet I can't see anything obvious from within logbook that can trigger this behavior, apart from basic imports from threading. Can you perhaps narrow it down to a more specific culprit?

nonamestreet commented 7 years ago

Yeah, if i set

has_gevent = False
use_gevent = False

in concurrency.py, they seems to work all right.

vmalloc commented 7 years ago

Ok. Will try to look into it ASAP. I'm a bit busy with other projects at the moment, so if you'd like to investigate and offer a PR that would be most welcome!