jupyter-xeus / xeus-python

Jupyter kernel for the Python programming language
BSD 3-Clause "New" or "Revised" License
439 stars 73 forks source link

xpython kernel does not execute tsfresh functions #230

Open nelsoncardenas opened 4 years ago

nelsoncardenas commented 4 years ago

I have installed Jupyter lab debugger in my windows 10.

I have followed explanation in: https://blog.jupyter.org/a-visual-debugger-for-jupyter-914e61716559

It was working ok. Now I'm executing tsfresh's function "extract_features", who works in python 3 kernel, but it doesn't in xpython kernel.

I let the code and the execution in both kernels.

from tsfresh import extract_features
settings = {'abs_energy': None,
            'standard_deviation': None,
            'absolute_sum_of_changes': None, 
            'number_crossing_m': [{'m': 0}]}
extracted_features = extract_features(DB.drop('Labels', axis=1), column_id="id", column_sort="time", default_fc_parameters=settings)
...

reporteXPython1 reporteXPython2

SylvainCorlay commented 4 years ago

Thanks for reporting.

To help us troubleshoot this, could you try disabling the progress bar by passing disable_progressbar=True to extract_features.

The issue may be with the tqdm progress bar.

SylvainCorlay commented 4 years ago

Hum, it does not seem that tqdm is at fault.

Tried tqdm in xeus-python and it just works:

from tqdm import tqdm
import time
for i in tqdm(range(1000)):
    time.sleep(0.01)
nelsoncardenas commented 4 years ago

I tried your solution with disable_progressbar=True. As you said, is not tqdm 's fault.

SylvainCorlay commented 4 years ago

I wonder if tsfresh tries to do some event loop integration with IPython that would fail due to the fact that this is not an IPython kernel.

Maybe @MaxBenChrist or @nils-braun have an idea.

nils-braun commented 4 years ago

Thanks for the mention!

So no, we do not interact with the event loop in ipython. "Basically" all we do is a groupby of the pandas dataframe and a multiprocessing call to the feature extractors.

I am wondering of the multiprocessing pool.imap_unordered call is the problem. Could you test with multiprocessing turned off (n_jobs=0)?