Closed MartinGuth closed 9 months ago
Hi Martin,
Thanks for this great first issue; clearly explained, and detailed.
This is true: "it seems that it actually tries to install the Python version of the code:". It works on Linux and macOS, but unfortunately, I use to encounter problems at the intersection R x Windows x reticulate. reticulate
does the translation from Python to R. Do you have it? I can't see it in your list of loaded packages. After that, we could investigate further.
This error is quite disturbing though: "No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils". What would be your take on that? (if you're into Python)
PS: When did you post your comment? I just commented, on Thursday, September 10 @ 10:35 am, Paris, and it seems to indicate that I did it 9 hours ago...
Hi Thierry,
sorry for the delay in replying!
I installed the rerticulate
package and tried installing the mlsauce package again, but it still gave me the same error. I then re-installed Python (v3.8.5/64 Bit)
with the default settings and paths - still got the error.
I went back to the error message in the R console and did tried solving the issues step-by-step:
Warning in system2(python, args) : '""' not found
- a bit of searching led me to this discussion py_config() unable to detect system-installed python in R-3.6.0 on Windows 10 and I thus included in my .Renviron file RETICULATE_PYTHON = "C:\\Users\\Martin\\AppData\\Local\\Programs\\Python\\Python38"
Next error Error: package or namespace load failed for 'mlsauce': .onLoad failed in loadNamespace() for 'mlsauce', details: call: NULL error: could not find a Python environment for C:/Users/Martin/AppData/Local/Programs/Python/Python38/python.exe
- I followed the basic setup for Python in RStudio Installing and Configuring Python with RStudio and setup a virtual environment (however not in my project, as I am not a regular Pyhton user, but straight in the base installation) and added another line to my .Renviron file RETICULATE_PYTHON_VENV = "C:\\Users\\Martin\\AppData\\Local\\Programs\\Python\\Python38\\python"
Because of distutils.errors.DistutilsPlatformError: Microsoft Visual C++ 14.0 is required.
- I installed the C++ build tools from https://visualstudio.microsoft.com/visual-cpp-build-tools/
But I am still stuck at:
> devtools::install_github("Techtonique/mlsauce/R-package")
Downloading GitHub repo Techtonique/mlsauce@HEAD
√ checking for file 'C:\Users\Martin\AppData\Local\Temp\RtmpM1j2Tq\remotes2280166744a9\Techtonique-mlsauce-db6ca8e\R-package/DESCRIPTION' ...
- preparing 'mlsauce':
√ checking DESCRIPTION meta-information ...
- checking for LF line-endings in source and make files and shell scripts
- checking for empty or unneeded directories
- building 'mlsauce_0.7.3.tar.gz'
Installing package into ‘C:/Users/Martin/Documents/R/win-library/4.0’
(as ‘lib’ is unspecified)
* installing *source* package 'mlsauce' ...
** using staged installation
** R
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
converting help for package 'mlsauce'
finding HTML links ... done
AdaOpt html
LSBoostClassifier html
LSBoostRegressor html
LassoRegressor html
RidgeRegressor html
StumpClassifier html
** building package indices
** testing if installed package can be loaded from temporary location
*** arch - i386
Warning in system2(python, args) : '""' not found
Error: package or namespace load failed for 'mlsauce':
.onLoad failed in loadNamespace() for 'mlsauce', details:
call: NULL
error: could not find a Python environment for C:/Users/Martin/AppData/Local/Programs/Python/Python38/python.exe
Error: loading failed
Execution halted
*** arch - x64
Warning in system2(python, args) : '""' not found
Error: package or namespace load failed for 'mlsauce':
.onLoad failed in loadNamespace() for 'mlsauce', details:
call: NULL
error: could not find a Python environment for C:/Users/Martin/AppData/Local/Programs/Python/Python38/python.exe
Error: loading failed
Execution halted
ERROR: loading failed for 'i386', 'x64'
* removing 'C:/Users/Martin/Documents/R/win-library/4.0/mlsauce'
Error: Failed to install 'mlsauce' from GitHub:
(converted from warning) installation of package ‘C:/Users/Martin/AppData/Local/Temp/RtmpM1j2Tq/file2280e8f1c1f/mlsauce_0.7.3.tar.gz’ had non-zero exit status
Any idea how to proceed?
P.S. you've answered my last comment right away :)
Hi Martin,
Would you try a few things mentioned in the following page?
https://rstudio.github.io/reticulate/articles/versions.html
Such as use_python
to specify the executable? Before installing. And check py_config()
, py_discover_config()
to see what is used.
Hi Martin, @MartinGuth,
How did it go?
Would you uninstall everything and try again now?
Starting from version 0.5, the package in Windows is not installed either through R or through Python. Finally, indicate in the project description that it is intended only for Linux. People are wasting their time pointlessly. Good luck
@VladPerervenko one constructive way would be to contribute a solution for Windows, if you know how to fix this. I'm not paid to create mlsauce.
As far as I remember, problems with version 5 were associated with the use of the JAX compiler. At that time (it was a long time ago), this compiler only worked under Linux. Now I see that it can be installed under Windows as well. True, dances with a tambourine will be required. If you haven't changed anything drastically and are still using JAX, I will try to install it on Windows and install your package. He is very interesting to me. Good luck
@VladPerervenko If it's interesting to you, then we can try to find a way. It'd be cool if you could post the error messages here (as a starting point). So that we can investigate.
When installing in Python, I get errors mlsauce08_Err.txt Maybe there is a problem here? ValueError: 'mlsauce\demo' is not a directory
I am not a pythonist. I cannot determine where and what the error is. sorry
@VladPerervenko, Ok, can you try again now, (pip install mlsauce --upgrade), and report the error message, if there's any? Please make sure that it's mlsauce's version 0.8.1.
Now it's installed in Python well.
Building wheels for collected packages: mlsauce Building wheel for mlsauce (setup.py) ... done Created wheel for mlsauce: filename=mlsauce-0.8.1-cp38-cp38-win_amd64.whl size=542256 sha256=b27228396c0080d4c4cb29000cb707136db71bc703bb76ab47dc7208a3be4175 Stored in directory: c:\users\user\appdata\local\pip\cache\wheels\bf\35\2e\850b79c36b943f82796f311027998884d1a37377d657f87be9 Successfully built mlsauce Installing collected packages: mlsauce Attempting uninstall: mlsauce Found existing installation: mlsauce 0.7.3 Uninstalling mlsauce-0.7.3: Successfully uninstalled mlsauce-0.7.3 Successfully installed mlsauce-0.8.1
When installing a package in R, will the environment in Python be detected automatically? Or do you need to activate it first? Good luck
I have not installed in R. Today I will check from Python via Reticulate.
The same story `Restarting R session...
reticulate::use_miniconda("mlsauce", required = TRUE) import("mlsauce")->mls Ошибка в import("mlsauce") :не могу найти функцию "import" require(reticulate) Загрузка требуемого пакета: reticulate py_config() python: C:/Users/User/AppData/Local/r-miniconda/envs/mlsauce/python.exe libpython: C:/Users/User/AppData/Local/r-miniconda/envs/mlsauce/python38.dll pythonhome: C:/Users/User/AppData/Local/r-miniconda/envs/mlsauce version: 3.8.3 (default, Jul 2 2020, 17:30:36) [MSC v.1916 64 bit (AMD64)] Architecture: 64bit numpy: C:/Users/User/AppData/Local/r-miniconda/envs/mlsauce/Lib/site-packages/numpy numpy_version: 1.18.5
NOTE: Python version was forced by use_python function
import("mlsauce")->mls Ошибка в py_module_import(module, convert = convert) : ModuleNotFoundError: No module named 'jax'
Detailed traceback:
File "C:\Users\User\Documents\R\win-library\4.0\reticulate\python\rpytools\loader.py", line 19, in _import_hook
module = _import(
File "C:\Users\User\AppData\Local\R-MINI~1\envs\mlsauce\lib\site-packages\mlsauce__init__.py", line 64, in
@VladPerervenko , thanks for your feedback. So, what would you suggest for R+Windows? For example, import jax conditionnally to the system (Windows, macOS, Linux), or something else? (e.g include your last comment in the README? etc.)
I see two options:
Installation sequence as here and here and here
PowerShell: https://github.com/PowerShell/PowerShell/releases/tag/v7.1.2
Bazel: https://github.com/bazelbuild/bazel/releases
MSYS2: https://repo.msys2.org/distrib/x86_64/msys2-x86_64-20210105.exe
Check if there is pacman in PATH
Install patch realpath: pacman -S patch realpath
Check that the PATH contains bazel, path, realpath
Clone the repository: git clone https://github.com/google/jax
We go to the directory
Activating the conda environment: conda activate
Trying to build jaxlib: python. \ Build \ build.py
If so, install jax: pip install -e.
Trying to run the package in Python
>>> import mlsauce as mls Traceback (most recent call last): File "<stdin>", line 1, in <module> File "C:\Users\User\AppData\Local\R-MINI~1\envs\mlsauce\lib\site-packages\mlsauce\__init__.py", line 64, in <module> from .booster import LSBoostClassifier, LSBoostRegressor File "C:\Users\User\AppData\Local\R-MINI~1\envs\mlsauce\lib\site-packages\mlsauce\booster\__init__.py", line 1, in <module> from ._booster_regressor import LSBoostRegressor File "C:\Users\User\AppData\Local\R-MINI~1\envs\mlsauce\lib\site-packages\mlsauce\booster\_booster_regressor.py", line 2, in <module> import jax.numpy as jnp File "c:\bin\jax\jax\__init__.py", line 22, in <module> from .api import ( File "c:\bin\jax\jax\api.py", line 39, in <module> from . import core File "c:\bin\jax\jax\core.py", line 31, in <module> from . import dtypes File "c:\bin\jax\jax\dtypes.py", line 32, in <module> from .lib import xla_client File "c:\bin\jax\jax\lib\__init__.py", line 53, in <module> from jaxlib import xla_client ImportError: cannot import name 'xla_client' from 'jaxlib' (unknown location)
Again a problem.
Obviously, to install the JAX library in Windows, you need a high qualification of a programmer. If any enthusiast can install this library, experience sharing please ..
Hi @VladPerervenko, I appreciate the effort, thanks. Could you give it a try now, please (version 0.8.2)? I was previously importing 'jax' at the wrong place, in a LSBoost file. Actually, you don't need to have jax installed on Windows yet.
`... Successfully built mlsauce Installing collected packages: mlsauce Successfully installed mlsauce-0.8.2 .....
(mlsauce) C:\Users\User>python Python 3.8.0 (default, Nov 6 2019, 16:00:02) [MSC v.1916 64 bit (AMD64)] :: Anaconda, Inc. on win32 Type "help", "copyright", "credits" or "license" for more information.
import mlsauce as mls Traceback (most recent call last): File "
", line 1, in File "C:\Users\User\AppData\Local\R-MINI~1\envs\mlsauce\lib\site-packages\mlsauce__init.py", line 64, in from .booster import LSBoostClassifier, LSBoostRegressor File "C:\Users\User\AppData\Local\R-MINI~1\envs\mlsauce\lib\site-packages\mlsauce\booster__init__.py", line 1, in init__.py", line 1, infrom ._booster_regressor import LSBoostRegressor File "C:\Users\User\AppData\Local\R-MINI~1\envs\mlsauce\lib\site-packages\mlsauce\booster_booster_regressor.py", line 6, in from . import _boosterc as boosterc File "mlsauce\booster_boosterc.pyx", line 16, in init mlsauce.booster._boosterc File "C:\Users\User\AppData\Local\R-MINI~1\envs\mlsauce\lib\site-packages\mlsauce\lasso\ from ._lasso import LassoRegressor File "C:\Users\User\AppData\Local\R-MINI~1\envs\mlsauce\lib\site-packages\mlsauce\lasso_lasso.py", line 8, in from . import _lassoc as mo File "mlsauce\lasso_lassoc.pyx", line 16, in init mlsauce.lasso._lassoc ModuleNotFoundError: No module named 'jax`
@VladPerervenko Yea... Anything 'jax' must be after 'if platform.system() in ('Linux', 'Darwin'):'
If you fix it, I'm ready to check.
@VladPerervenko I just made the change, you can check. Thanks!
The version has not changed? 0.8.2?
@VladPerervenko it's 0.8.3 now
`(mlsauce) C:\Users\User>python Python 3.8.0 (default, Nov 6 2019, 16:00:02) [MSC v.1916 64 bit (AMD64)] :: Anaconda, Inc. on win32 Type "help", "copyright", "credits" or "license" for more information.
import mlsauce as mls ` I need to check in R. I need a little time.
Works in P using the Python package. `> reticulate::use_miniconda("mlsauce", required = TRUE)
reticulate::py_config() python: C:/Users/User/AppData/Local/r-miniconda/envs/mlsauce/python.exe libpython: C:/Users/User/AppData/Local/r-miniconda/envs/mlsauce/python38.dll pythonhome: C:/Users/User/AppData/Local/r-miniconda/envs/mlsauce version: 3.8.3 (default, Jul 2 2020, 17:30:36) [MSC v.1916 64 bit (AMD64)] Architecture: 64bit numpy: C:/Users/User/AppData/Local/r-miniconda/envs/mlsauce/Lib/site-packages/numpy numpy_version: 1.18.5
NOTE: Python version was forced by use_python function
require(reticulate) mls <- import("mlsauce") Input data X <- as.matrix(MASS::Boston[, -1]) y <- as.integer(MASS::Boston[, 1])
n <- dim(X)[1] p <- dim(X)[2]
number of repeats for obtaining the distribution of errors n_repeats <- 100 function for calculating the out-of-sample error, based on activation functions get_rmse_error <- function(activation = c("sigmoid", "tanh", "relu6", "relu")) { err <- rep(0, n_repeats) pb <- txtProgressBar(min = 0, max = n_repeats, style = 3) for (i in 1:n_repeats) { set.seed(21341+i10) train_index <- sample(x = 1:n, size = floor(0.8n), replace = TRUE) test_index <- -train_index X_train <- as.matrix(X[train_index, ]) y_train <- as.double(y[train_index]) X_test <- as.matrix(X[test_index, ]) y_test <- as.double(y[test_index]) using default parameters obj <- mls::LSBoostRegressor(verbose = FALSE, activation = match.arg(activation)) obj$fit(X_train, y_train) err[i] <- sqrt(mean((obj$predict(X_test) - y_test)2)) setTxtProgressBar(pb, i) } return(err) } test set error for g=sigmoid (err1 <- get_rmse_error("sigmoid")) ==================================================================================| 100% [1] 8.109019 7.633484 7.013467 7.902473 6.983382 6.890838 7.584176 8.526518 5.811761 4.882274 [11] 8.279541 5.504811 6.608797 3.630142 5.504503 8.120792 7.524510 7.686959 7.822513 6.606192 [21] 5.380453 6.679731 8.092776 6.557862 9.163628 4.521416 6.864968 7.164130 5.218686 6.300976 [31] 4.426167 7.628373 8.296893 5.681510 6.286927 6.920092 7.959753 4.444488 7.605373 6.015300 [41] 8.331245 4.309038 6.232670 5.236483 8.089969 8.420141 5.752030 5.888024 5.661586 5.759942 [51] 6.380020 7.804472 4.669252 5.742645 5.667491 6.350312 6.538349 6.435074 5.671562 7.034257 [61] 7.249832 5.419883 5.221553 5.377361 6.858819 6.080643 6.050147 4.692001 5.334371 4.927046 [71] 5.350460 6.414981 6.366472 7.286016 5.456898 5.849108 7.457750 5.993626 5.545376 4.609542 [81] 7.277193 6.527692 7.525783 8.223626 4.916341 6.145885 7.981039 5.247310 5.375255 7.670348 [91] 7.518121 5.746653 7.648944 7.820192 6.725242 7.655622 5.758472 7.942319 5.260720 8.050118 test set error for g=tanh** (err2 <- get_rmse_error("tanh")) ==================================================================================| 100% [1] 7.868073 7.614182 6.936074 7.926574 7.034764 6.968830 7.596908 8.539407 5.785825 5.215545 [11] 8.241377 5.631660 6.611435 3.570719 5.546593 8.090995 7.501663 7.652137 7.695753 6.680120 [21] 5.366591 6.645411 8.007549 6.608058 9.198955 4.390428 6.741804 7.119123 5.189714 6.290484 [31] 4.477055 7.638301 8.253871 5.719206 6.126541 6.955791 7.982074 4.406688 7.615751 5.957745 [41] 8.359519 4.169886 6.245351 5.264157 8.068617 8.420823 5.775245 5.642445 5.780387 5.916511 [51] 6.376110 7.761837 4.630182 5.708699 5.733793 6.054252 6.518859 6.403737 5.607429 7.030983 [61] 7.222176 5.535944 5.284809 5.314265 6.916537 5.922615 6.107705 4.565625 5.343445 4.931251 [71] 5.283927 6.476644 6.224258 7.246286 5.438388 5.830465 7.400990 6.004810 5.376800 4.640129 [81] 6.961114 6.459700 7.557108 8.204921 4.959318 6.111345 7.950669 5.192580 5.415752 7.684638 [91] 7.588308 5.504545 7.649933 7.801889 6.705058 7.686048 5.725376 7.979240 5.282672 7.996989 par(mfrow=c(1, 2)) hist(err1, main = "distribution of test set error \n (activation = sigmoid)") hist(err2, main = "distribution of test set error \n (activation = tanh)")
print(sessionInfo()) R version 4.0.3 (2020-10-10) Platform: x86_64-w64-mingw32/x64 (64-bit) Running under: Windows 10 x64 (build 19042)
Matrix products: default
locale: [1] LC_COLLATE=Russian_Russia.1251 LC_CTYPE=Russian_Russia.1251 LC_MONETARY=Russian_Russia.1251 [4] LC_NUMERIC=C LC_TIME=Russian_Russia.1251
attached base packages: [1] stats graphics grDevices utils datasets methods base
other attached packages: [1] reticulate_1.18
loaded via a namespace (and not attached):
[1] MASS_7.3-53 compiler_4.0.3 Matrix_1.2-18 tools_4.0.3 rappdirs_0.3.3 yaml_2.2.1
[7] Rcpp_1.0.6 grid_4.0.3 jsonlite_1.7.2 lattice_0.20-41`
How to install a package in R, You need to think. In python, a package "mlsauce" can already be installed in any environment. I have it set to env/ mlsauce/
@VladPerervenko so, it works on your Windows machine now, right?
I don't get this part: '> How to install a package in R, You need to think. In python, a package "mlsauce" can already be installed in any environment. I have it set to env/ mlsauce/'
@VladPerervenko and @MartinGuth: Shall we close the issue now? @VladPerervenko Would you like to guest blog this on my page https://thierrymoudiki.github.io/blog/? ('How to install on Windows')
I think it's early to close. Have someone install the package in R according to your instructions. For example @MartinGuth . If everything works out, then close it.
What exactly did you want and what did I post on your blog. My English through a translator is not very clear. If only codes with short comments.
@VladPerervenko Yes, code with short comments. And I'll do the rest.
I tried to install mlsause (0.8.3). Mistake `> devtools::install_github("Techtonique/mlsauce/R-package") Downloading GitHub repo Techtonique/mlsauce@HEAD √ checking for file 'C:\Users\User\AppData\Local\Temp\RtmpkNQwm6\remotes19f4b2d3c67\Techtonique-mlsauce-3f82d4f\R-package/DESCRIPTION' ...
Устанавливаю пакет в ‘C:/Users/User/Documents/R/win-library/4.1’ (потому что ‘lib’ не определено)
I'm trying to install querier:
(r-reticulate) C:\Users\User>pip install querier
Traceback (most recent call last):
File "C:\Users\User\AppData\Local\R-MINI~1\envs\r-reticulate\Scripts\pip-script.py", line 6, in
Probably a problem with the brotli module. By the way, the same error when trying to install nnetsauce in R. But unlike nnetsauce, mlsauce is installed in Python and works fine in R through reticulate.
@VladPerervenko at the end of your message, you say "unlike nnetsauce, mlsauce is installed in Python and works fine in R through reticulate.". But what did you say at the beginning of the message? That installing on Windows didn't work?
I can't read the following error messages:
1) Предупреждение в system2(python, args) : '""' not found
2) AttributeError: module 'brotli' has no attribute 'error' Error: не удалась загрузка пакета или пространства имен для 'mlsauce': .onLoad не удалось в loadNamespace() для 'mlsauce', подробности: вызов: NULL ошибка: Error installing package(s): "querier" Ошибка: loading failed Выполнение остановлено
__What do they mean in english?__
PS: I've just tried to install both mlsauce and nnetsauce in a fresh environment on RStudio Cloud (Ubuntu OS), and it works
reticulate::py_config() python: C:/Users/User/AppData/Local/r-miniconda/envs/mlsauce/python.exe libpython: C:/Users/User/AppData/Local/r-miniconda/envs/mlsauce/python38.dll pythonhome: C:/Users/User/AppData/Local/r-miniconda/envs/mlsauce version: 3.8.3 (default, Jul 2 2020, 17:30:36) [MSC v.1916 64 bit (AMD64)] Architecture: 64bit numpy: C:/Users/User/AppData/Local/r-miniconda/envs/mlsauce/Lib/site-packages/numpy numpy_version: 1.18.5
NOTE: Python version was forced by use_python function
require(reticulate) Загрузка требуемого пакета: reticulate
mls <- import("mlsauce") library(datasets)
X <- as.matrix(datasets::mtcars[, -1]) y <- as.integer(datasets::mtcars[, 1])
n <- dim(X)[1] p <- dim(X)[2] set.seed(21341) train_index <- sample(x = 1:n, size = floor(0.8*n), replace = TRUE) test_index <- -train_index X_train <- as.matrix(X[train_index, ]) y_train <- as.double(y[train_index]) X_test <- as.matrix(X[test_index, ]) y_test <- as.double(y[test_index]) obj <- mls$LSBoostRegressor()
start <- proc.time()[3] obj$fit(X_train, y_train) 100%|██████████| 100/100 [00:00<00:00, 787.37it/s]LSBoostRegressor()
print(proc.time()[3] - start) elapsed 0.16 start <- proc.time()[3] print(sqrt(mean((obj$predict(X_test) - y_test)**2))) [1] 6.482376 print(proc.time()[3] - start) elapsed 0.03 `
error messages:
summary:
Hi Thierry,
thank you for providing the code for the LSBoost algorithm. I just tried to install the R-package, but it is not working!
I followed the steps outlined:
However, it seems that it actually tries to install the Python version of the code:
Then the installation carries on collection Python packages, but it fails at executing Python.exe:
It tries a few times more to access the Python.exe and finally exits the installation in the R console with an ultimate error message:
Any idea how to fix this issue? Finally my R session info:
Best, Martin