ryankarel
commented
5 years ago I should also mention that I'm using Windows 10.
Open ryankarel opened 5 years ago
ryankarel
commented
5 years ago I should also mention that I'm using Windows 10.
ercbk
commented
4 years ago In issue 134 @jjallaire thinks it may be a problem with multiprocessing and embedded python interpreters, but that was back in 2017. Really surprised this problem isn't mentioned anywhere in the documentation or this issue hasn't received a response. Kind of a big deal.
kevinushey
commented
4 years ago For this particular issue, the problem is that sys$executable and sys$_base_executable do not point at the expected things for joblib:
library(reticulate)
sys <- import("sys")
print(sys$executable)
## [1] "C:\\R\\R-36~1.1\\bin\\x64\\Rterm.exe"
print(sys$`_base_executable`)
## [1] "C:\\R\\R-36~1.1\\bin\\x64\\Rterm.exe"I think we need to set these to the path to the Python interpreter for multiprocessing modules / joblib to work as expected.
skeydan
commented
4 years ago (((Aside question, is this expected to be a Windows-only problem? For me the above code works fine under Linux. But for me, sys$executable (https://github.com/rstudio/reticulate/blob/9dedfb3fa043ed5f035d6a1ee162cc9ee3b1c64e/R/repl.R#L265) points to Python, not R...)))
ercbk
commented
4 years ago I think I'm encountering the same issue, but I'm just using a R script and not trying to source and knit a Rmd. So, I'll add my error report to the mix in case it helps any. Btw if there's a hack to get around this in the meantime, I'd appreciate that. Thank you
pacman::p_load(dials, reticulate)
sk_e <- import("sklearn.ensemble")
sk_ms <- import("sklearn.model_selection")
sim_data <- function(n) {
tmp <- mlbench::mlbench.friedman1(n, sd=1)
tmp <- cbind(tmp$x, tmp$y)
tmp <- as.data.frame(tmp)
names(tmp)[ncol(tmp)] <- "y"
tmp
}
dat <- sim_data(10000)
pdat = r_to_py(dat)
y = pdat$pop('y')$values
X <- pdat
rf_est <- sk_e$RandomForestRegressor(criterion = "mae", random_state = 1L)
# rf_est <- sk_e$RandomForestRegressor(criterion = "mae", n_jobs = -1L, random_state = 1L)
rf_params <- r_to_py(dials::grid_latin_hypercube(
mtry(range = c(3, 4)),
trees(range = c(200, 300)),
size = 40
))
max_features <- rf_params$pop('mtry')$values
n_estimators <- rf_params$pop('trees')$values
rf_grid <- py_dict(list('max_features', 'n_estimators'), list(max_features, n_estimators))
cv <- sk_ms$RepeatedKFold(n_splits = 2L,
n_repeats = 2L,
random_state = 1L)
mod_select <- sk_ms$GridSearchCV(estimator = rf_est,
param_grid = rf_grid,
scoring = 'neg_mean_absolute_error',
cv = cv,
n_jobs = -1L,
refit = TRUE)
results <- mod_select$fit(X, y)
#> Error in py_call_impl(callable, dots$args, dots$keywords): OSError: [Errno 22] Invalid argument
#>
#> Detailed traceback:
#> File "C:\Users\tbats\Miniconda3\lib\site-packages\sklearn\model_selection\_search.py", line 710, in fit
#> self._run_search(evaluate_candidates)
#> File "C:\Users\tbats\Miniconda3\lib\site-packages\sklearn\model_selection\_search.py", line 1151, in _run_search
#> evaluate_candidates(ParameterGrid(self.param_grid))
#> File "C:\Users\tbats\Miniconda3\lib\site-packages\sklearn\model_selection\_search.py", line 689, in evaluate_candidates
#> cv.split(X, y, groups)))
#> File "C:\Users\tbats\Miniconda3\lib\site-packages\joblib\parallel.py", line 1004, in __call__
#> if self.dispatch_one_batch(iterator):
#> File "C:\Users\tbats\Miniconda3\lib\site-packages\joblib\parallel.py", line 835, in dispatch_one_batch
#> self._dispatch(tasks)
#> File "C:\Users\tbats\Miniconda3\lib\site-packages\joblib\parallel.py", line 754, in _dispatch
#> job = self._backend.apply_async(batch, callback=cb)
#> File "C:\Users\tbats\Miniconda3\lib\site-packages\joblib\_parallel_backends.py", line 551, in apply_async
#> future = self._workers.submit(SafeFunction(func))
#> File "C:\Users\tbats\Miniconda3\lib\site-packages\joblib\externals\loky\reusable_executor.py", line 160, in submit
#> fn, *args, **kwargs)
#> File "C:\Users\tbats\Miniconda3\lib\site-packages\joblib\externals\loky\process_executor.py", line 1047, in submit
#> self._ensure_executor_running()
#> File "C:\Users\tbats\Miniconda3\lib\site-packages\joblib\externals\loky\process_executor.py", line 1021, in _ensure_executor_running
#> self._adjust_process_count()
#> File "C:\Users\tbats\Miniconda3\lib\site-packages\joblib\externals\loky\process_executor.py", line 1012, in _adjust_process_count
#> p.start()
#> File "C:\Users\tbats\Miniconda3\lib\multiprocessing\process.py", line 112, in start
#> self._popen = self._Popen(self)
#> File "C:\Users\tbats\Miniconda3\lib\site-packages\joblib\externals\loky\backend\process.py", line 39, in _Popen
#> return Popen(process_obj)
#> File "C:\Users\tbats\Miniconda3\lib\site-packages\joblib\externals\loky\backend\popen_loky_win32.py", line 55, in __init__
#> process_obj._name, getattr(process_obj, "init_main_module", True))
#> File "C:\Users\tbats\Miniconda3\lib\site-packages\joblib\externals\loky\backend\spawn.py", line 86, in get_preparation_data
#> _resource_tracker.ensure_running()
#> File "C:\Users\tbats\Miniconda3\lib\site-packages\joblib\externals\loky\backend\resource_tracker.py", line 83, in ensure_running
#> if self._check_alive():
#> File "C:\Users\tbats\Miniconda3\lib\site-packages\joblib\externals\loky\backend\resource_tracker.py", line 163, in _check_alive
#> self._send('PROBE', '', '')
#> File "C:\Users\tbats\Miniconda3\lib\site-packages\joblib\externals\loky\backend\resource_tracker.py", line 185, in _send
#> nbytes = os.write(self._fd, msg)Created on 2020-01-30 by the reprex package (v0.3.0)
kevinushey
commented
4 years ago What worked for me was to re-assign those values in the sys and multiprocessing modules. That is, you can try running this at the top of your script (before importing any other modules):
library(reticulate)
# update executable path in sys module
sys <- import("sys")
exe <- file.path(sys$exec_prefix, "pythonw.exe")
sys$executable <- exe
sys$`_base_executable` <- exe
# update executable path in multiprocessing module
multiprocessing <- import("multiprocessing")
multiprocessing$set_executable(exe)Worked! Thanks again. One more question - will the pythonw.exe instances eventually end on their own or should I terminate them? I had to end the first session I tried this, and those pythonw.exe instances are still around along with the ones from this second run.
Nevermind. They ended when I quit RStudio.
kevinushey
commented
4 years ago Yes indeed, they'll exit when the R session is shut down. (I'm not sure whether Python tries to re-use the existing child sessions, or if they should normally be shut down after running the requisite code, though.)
ercbk
commented
4 years ago Ran a similar script overnight through RScript.exe. It's been over 3.5 hrs since the job finished, and the python instances didn't terminate. So, I guess these might need to be shut down.
vermosen
commented
3 years ago Hi,
I see a similar problem in centos 8 using both base R and Rstudio. Consider the following Rscript:
library(reticulate)
use_condaenv(condaenv = 'py38', conda = '/opt/miniconda/bin/conda')
sk <- NULL
sk$ds <- import("sklearn.datasets")
sk$ms <- import("sklearn.model_selection")
sk$da <- import("sklearn.discriminant_analysis")
# define dataset
data <- sk$ds$make_classification(n_samples=1000L, n_features=10L, n_informative=10L, n_redundant=0L, random_state=1L)
# define model
model <- sk$da$LinearDiscriminantAnalysis()
# define model evaluation method
cv <- sk$ms$RepeatedStratifiedKFold(n_splits=10L, n_repeats=3L, random_state=1L)
# evaluate the model
scores <- sk$ms$cross_val_score(model, data[[1]], data[[2]]
, scoring='accuracy', cv=cv, n_jobs=2L)
# Error: C stack usage 331680283904 is too close to the limit
cat('score: ', score)as soon the n_jobs value create forks (i.e. n_jobs != 1), the R session crashes on the last line with the message above. Also, I end up with stuck python processes I have to kill manually.
I checked the executable and _base_executable values mentioned and there are both set to the correct python binary.
Here is my R setup:
R version 4.0.3 (2020-10-10)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: CentOS Linux 8
Matrix products: default
BLAS/LAPACK: /opt/r40/lib64/libopenblas.so.0.3
locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
[3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
[7] LC_PAPER=en_US.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] reticulate_1.18
loaded via a namespace (and not attached):
[1] compiler_4.0.3 Matrix_1.2-18 Rcpp_1.0.5 grid_4.0.3
[5] jsonlite_1.7.1 lattice_0.20-41
shivam7898
commented
2 years ago Reiterating this comment above, because it took me sometime, to do the same in a Python chunk or file in Windows 10
import sys, os, multiprocessing
q_EXE_PATH = os.path.join(sys.exec_prefix, 'pythonw.exe')
sys.executable = q_EXE_PATH
sys._base_executable = q_EXE_PATH
multiprocessing.set_executable(q_EXE_PATH)
Here's the code I'm using:
When I try to knit an RMD file with the above python code chunk, a terminal window appears with the text:
Here's my (abbreviated)
py_config()output:As mentioned above, running the python script in a Python IDE works perfectly, but running it using the reticulate package with Rstudio results in an indefinitely hanging process. Is reticulate not meant to run multiple jobs, or is this a bug?