Open leummas opened 5 months ago
I am also getting a similar error, with both classification and regression functions. Looking at my activity monitor, I notice the user CPU shoots up to close to 100% right before the error triggers. Something isn't quite right with the parallel jobs code.
Running the example code Im receiving the following error:
_RemoteTraceback Traceback (most recent call last) _RemoteTraceback: """ Traceback (most recent call last): File "/opt/conda/envs/feature_creation_env/lib/python3.12/site-packages/joblib/externals/loky/process_executor.py", line 426, in _process_worker call_item = call_queue.get(block=True, timeout=timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/envs/feature_creation_env/lib/python3.12/multiprocessing/queues.py", line 122, in get return _ForkingPickler.loads(res) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/envs/feature_creation_env/lib/python3.12/site-packages/mrmr/init.py", line 1, in
from . import bigquery
File "/opt/conda/envs/feature_creation_env/lib/python3.12/site-packages/mrmr/bigquery.py", line 3, in
from .main import mrmr_base, groupstats2fstat
File "/opt/conda/envs/feature_creation_env/lib/python3.12/site-packages/mrmr/main.py", line 1, in
import pandas as pd
File "/opt/conda/envs/feature_creation_env/lib/python3.12/site-packages/pandas/init.py", line 46, in
from pandas.core.api import (
File "/opt/conda/envs/feature_creation_env/lib/python3.12/site-packages/pandas/core/api.py", line 47, in
from pandas.core.groupby import (
File "/opt/conda/envs/feature_creation_env/lib/python3.12/site-packages/pandas/core/groupby/init.py", line 1, in
from pandas.core.groupby.generic import (
File "/opt/conda/envs/feature_creation_env/lib/python3.12/site-packages/pandas/core/groupby/generic.py", line 67, in
from pandas.core.frame import DataFrame
...
--> 754 raise self._result
755 return self._result
756 finally:
BrokenProcessPool: A task has failed to un-serialize. Please ensure that the arguments of the function are all picklable.