smazzanti / mrmr

mRMR (minimum-Redundancy-Maximum-Relevance) for automatic feature selection at scale.
MIT License
531 stars 79 forks source link

Parallel Issue #26

Closed christyohanes closed 6 months ago

christyohanes commented 1 year ago

~\anaconda3\lib\site-packages\mrmr\main.py in mrmr_base(K, relevance_func, redundancy_func, relevance_args, redundancy_args, denominator_func, only_same_domain, return_scores, show_progress) 96 """ 97 ---> 98 relevance = relevance_func(**relevance_args) 99 features = relevance[relevance.fillna(0) > 0].index.to_list() 100 relevance = relevance.loc[features]

~\anaconda3\lib\site-packages\mrmr\pandas.py in f_classif(X, y, n_jobs) 43 def f_classif(X, y, n_jobs): 44 return parallel_df(_f_classif, X, y, n_jobs=n_jobs) ---> 45 46 47 def f_regression(X, y, n_jobs):

~\anaconda3\lib\site-packages\mrmr\pandas.py in parallel_df(func, df, series, n_jobs) 17 delayed(func)(df.iloc[:, col_chunk], series) 18 for col_chunk in col_chunks ---> 19 ) 20 return pd.concat(lst) 21

~\anaconda3\lib\site-packages\joblib\parallel.py in call(self, iterable) 1096 1097 with self._backend.retrieval_context(): -> 1098 self.retrieve() 1099 # Make sure that we get a last message telling us we are done 1100 elapsed_time = time.time() - self._start_time

~\anaconda3\lib\site-packages\joblib\parallel.py in retrieve(self) 973 try: 974 if getattr(self._backend, 'supports_timeout', False): --> 975 self._output.extend(job.get(timeout=self.timeout)) 976 else: 977 self._output.extend(job.get())

~\anaconda3\lib\site-packages\joblib_parallel_backends.py in wrap_future_result(future, timeout) 565 AsyncResults.get from multiprocessing.""" 566 try: --> 567 return future.result(timeout=timeout) 568 except CfTimeoutError as e: 569 raise TimeoutError from e

~\anaconda3\lib\concurrent\futures_base.py in result(self, timeout) 433 raise CancelledError() 434 elif self._state == FINISHED: --> 435 return self.__get_result() 436 else: 437 raise TimeoutError()

~\anaconda3\lib\concurrent\futures_base.py in __get_result(self) 382 def __get_result(self): 383 if self._exception: --> 384 raise self._exception 385 else: 386 return self._result

TerminatedWorkerError: A worker process managed by the executor was unexpectedly terminated. This could be caused by a segmentation fault while calling the function or by an excessive memory usage causing the Operating System to kill the worker.

smazzanti commented 6 months ago

If your machine does not allow to run jobs in parallel, it throws an exception as you got. Please try with 'n_jobs=1'