Error when running with multiple jobs: unexpected keyword argument 'target_offload'

[Bouncner]

I have been using your library for quite a while and am super happy with it. So first, thanks a lot!

Lately, I used my framework (which also uses your library) on modern many core server with many jobs. Worked fine. Now I have updated everything via pip and with 8 jobs on my MacBook, I got the following error.

This error does not occur when using only a single job (I pass the number of jobs to n_jobs).

I cannot nail the down the actual problem, but since it occurred right after the upgrade, I assume this might be the reason?

Am I doing something wrong here?

"""
Traceback (most recent call last):
  File "/Users/martin/opt/anaconda3/lib/python3.7/site-packages/joblib/externals/loky/process_executor.py", line 436, in _process_worker
    r = call_item()
  File "/Users/martin/opt/anaconda3/lib/python3.7/site-packages/joblib/externals/loky/process_executor.py", line 288, in __call__
    return self.fn(*self.args, **self.kwargs)
  File "/Users/martin/opt/anaconda3/lib/python3.7/site-packages/joblib/_parallel_backends.py", line 595, in __call__
    return self.func(*args, **kwargs)
  File "/Users/martin/opt/anaconda3/lib/python3.7/site-packages/joblib/parallel.py", line 263, in __call__
    for func, args, kwargs in self.items]
  File "/Users/martin/opt/anaconda3/lib/python3.7/site-packages/joblib/parallel.py", line 263, in <listcomp>
    for func, args, kwargs in self.items]
  File "/Users/martin/opt/anaconda3/lib/python3.7/site-packages/lineartree/_classes.py", line 56, in __call__
    with config_context(**self.config):
  File "/Users/martin/opt/anaconda3/lib/python3.7/contextlib.py", line 239, in helper
    return _GeneratorContextManager(func, args, kwds)
  File "/Users/martin/opt/anaconda3/lib/python3.7/contextlib.py", line 82, in __init__
    self.gen = func(*args, **kwds)
TypeError: config_context() got an unexpected keyword argument 'target_offload'
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "compression_selection_pipeline.py", line 41, in <module>
    model_pipeline.learn_runtime_models(calibration_result_dir)
  File "/Users/martin/Programming/compression_selection_v3/hyrise_calibration/model_pipeline.py", line 670, in learn_runtime_models
    non_splitting_models("table_scan", table_scans)
  File "/Users/martin/Programming/compression_selection_v3/hyrise_calibration/model_pipeline.py", line 590, in non_splitting_models
    fitted_model = model_dict["model"].fit(X_train, y_train)
  File "/Users/martin/Programming/compression_selection_v3/hyrise_calibration/model_pipeline.py", line 209, in fit
    return self.regression.fit(X, y)
  File "/Users/martin/opt/anaconda3/lib/python3.7/site-packages/lineartree/lineartree.py", line 187, in fit
    self._fit(X, y, sample_weight)
  File "/Users/martin/opt/anaconda3/lib/python3.7/site-packages/lineartree/_classes.py", line 576, in _fit
    self._grow(X, y, sample_weight)
  File "/Users/martin/opt/anaconda3/lib/python3.7/site-packages/lineartree/_classes.py", line 387, in _grow
    loss=loss)
  File "/Users/martin/opt/anaconda3/lib/python3.7/site-packages/lineartree/_classes.py", line 285, in _split
    for feat in split_feat)
  File "/Users/martin/opt/anaconda3/lib/python3.7/site-packages/joblib/parallel.py", line 1056, in __call__
    self.retrieve()
  File "/Users/martin/opt/anaconda3/lib/python3.7/site-packages/joblib/parallel.py", line 935, in retrieve
    self._output.extend(job.get(timeout=self.timeout))
  File "/Users/martin/opt/anaconda3/lib/python3.7/site-packages/joblib/_parallel_backends.py", line 542, in wrap_future_result
    return future.result(timeout=timeout)
  File "/Users/martin/opt/anaconda3/lib/python3.7/concurrent/futures/_base.py", line 435, in result
    return self.__get_result()
  File "/Users/martin/opt/anaconda3/lib/python3.7/concurrent/futures/_base.py", line 384, in __get_result
    raise self._exception
TypeError: config_context() got an unexpected keyword argument 'target_offload'

PS: I have already left a star. :D

[cerlymarco]

Hi, could u please provide a reproducible example?

[cerlymarco]

In this example, you can find a dummy trial using n_jobs=-1 and the latest version of linear-tree without problems

[Bouncner]

I don't know when I'll find the time to get a reproducable example. I'll reopen once I have some more information. I got the error multiple times, but not every time. I probably first need to ensure my updated setup runs as expected.