Setting number of threads not working on Windows

enweg commented 1 year ago

I am trying to set the number of threads to be used by JuliaCall via the code below:

Sys.setenv(JULIA_NUM_THREADS = 2)
# Sys.setenv(JULIA_HOME = "/Applications/Julia-1.8.app/Contents/Resources/julia/bin")
library(JuliaCall)
julia_command("Threads.nthreads()")
# On Mac this returns 2
# On Windows this returns 1

On my Mac machine (that's why there is a JULIA_HOME line, this works all well, but on my Windows machine JuliaCall always only uses a single thread. Is there some other way to set the number of threads on Windows that I am not aware of?

e-kotov commented 1 year ago

Same here.

@enweg are you admin/root on Windows?

enweg commented 1 year ago

@e-kotov Unfortunately not on the Windows machine. Does it work with admin/root rights?

e-kotov commented 1 year ago

@enweg at the moment I only have access to a Windows machine where I do not have admin rights, so I cannot test.

pteridin commented 7 months ago

Can reproduce. Though, setting the JULIA_NUM_THREADS globally as an admin works and setting JULIA_NUM_THREADS locally using the command line is a workaround too:

Open command line and set the number of threads, then open RStudio using the console:

set JULIA_NUM_THREADS=8
RStudio

Result:

> library(JuliaCall)
Warning message:
package ‘JuliaCall’ was built under R version 4.3.2 
> julia_command("Threads.nthreads()")
Julia version 1.10.0 at location C:\Users\xyz\.julia\juliaup\julia-1.10.0+0.x64.w64.mingw32\bin will be used.
Loading setup script for JuliaCall...
Finish loading setup script for JuliaCall.
8

It would still be useful to be able to use Sys.setenv(JULIA_NUM_THREADS = 2) or set the number of threads as a parameter to julia_setup.

edwardlavender commented 1 month ago

These are two work arounds on Windows that worked for me:

Use `set JULIA_NUM_THREADS` (as described above)

Launch CMD as administrator, set JULIA_NUM_THREADS in CMD and launch R/RStudio from the same CMD session:

In CMD:

set JULIA_NUM_THREADS = 4
cd "C:\Program Files\RStudio"
rstudio.exe

Then in R:

> library(JuliaCall)
> julia_command("Threads.nthreads()")
Julia version 1.6.7 at location C:\Users\Ian\AppData\Local\Programs\JULIA-~1.7\bin will be used.
Loading setup script for JuliaCall...
Finish loading setup script for JuliaCall.
4

This only works if you launch CMD as an administrator and if the environment variable JULIA_NUM_THREADS is set in the environment from which the Julia application starts (hence, the need to launch R/RStudio from the same CMD session).

Set system-wide environment variables

To set JULIA_NUM_THREADS for all Julia sessions:

Open the Start menu and search for "Environment Variables".
Select "Edit the system environment variables".
In the System Properties window, in the Advanced pane, click on the "Environment Variables" button.
In the Environment Variables window, under "System variables", click "New".
Set the variable name to JULIA_NUM_THREADS and the value as required.
OK everything and close the windows.

Now you can launch R/RStudio in the usual way (e.g., via an application shortcut).

Then in R:

> library(JuliaCall)
> julia_command("Threads.nthreads()")
Julia version 1.6.7 at location C:\Users\Ian\AppData\Local\Programs\JULIA-~1.7\bin will be used.
Loading setup script for JuliaCall...
Finish loading setup script for JuliaCall.
4

pteridin commented 1 month ago

As mentioned before: The workaround as admin does work. Thank you for providing detailled instructons @edwardlavender

Though it would be still nice to set the number of threads without admin rights and on a per-analysis basis. I am currently working on a multi user system environment where it would be sometimes beneficial to do some parallel number crunching on a bunch of CPU cores, but most of the time it would just be enough to use 1-4 cores.

Non-Contradiction / JuliaCall