JuliaLang / julia

The Julia Programming Language
https://julialang.org/
MIT License
45.87k stars 5.49k forks source link

JULIA_COPY_STACKS=1 seems to disable threading on 1.8.2 #47591

Open DrChainsaw opened 2 years ago

DrChainsaw commented 2 years ago

I don't have need for this to work, just trying to be a good open source citizen and reporting what looks like a regression. Sorry if it is just noise. I just happened to have this set for other (now obsolete) reasons and on 1.8.2 it triggered this exception. Below is a dependency free MWE.

$ export JULIA_COPY_STACKS=1; julia -t 4
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org/
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.8.2 (2022-09-29)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |
julia> let ths = zeros(Int, 10000)
              Threads.@threads for i in 1:length(ths) 
                  ths[i] = Threads.threadid() 
              end
              unique(ths) 
           end
1-element Vector{Int64}:
 1

julia> versioninfo()
Julia Version 1.8.2
Commit 36034abf260 (2022-09-29 15:21 UTC)
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 4 × Intel(R) Xeon(R) Gold 6132 CPU @ 2.60GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-13.0.1 (ORCJIT, skylake-avx512)
  Threads: 4 on 4 virtual cores
Environment:
  JULIA_COPY_STACKS = 1
$ export JULIA_COPY_STACKS=0; julia -t 4
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org/
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.8.2 (2022-09-29)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |
julia> let ths = zeros(Int, 10000)
              Threads.@threads for i in 1:length(ths) 
                  ths[i] = Threads.threadid() 
              end
              unique(ths) 
           end
4-element Vector{Int64}:
 3
 1
 4
 2
$ module add julia/1.6.1
$ export JULIA_COPY_STACKS=1; julia -t 4
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org/
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.6.1 (2021-04-23)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |
julia> let ths = zeros(Int, 10000)
              Threads.@threads for i in 1:length(ths) 
                  ths[i] = Threads.threadid() 
              end
              unique(ths) 
           end
4-element Vector{Int64}:
 1
 2
 3
 4
vtjnash commented 1 year ago

JULIA_COPY_STACKS=1 is discouraged, and forces every Task to stay on the thread that spawned it

chenspc commented 10 months ago

JULIA_COPY_STACKS=1 is discouraged, and forces every Task to stay on the thread that spawned it

Is it mentioned somewhere that this is discouraged apart from this issue? I cannot find this variable in Environment Variables, and according to #43463 (from two years ago) there wasn’t any documentation.

I had some multi-threaded code that worked fine before but dramatically slowed down after I used a package that required JavaCall.jl, which listed setting JULIA_COPY_STACKS to 1 as one of the setup steps. Because I did not noticed the slowdown immediately, it was very difficult to link these two things together. It would be nice if some sort of warning could be added when using Threads.@threads while JULIA_COPY_STACKS=1.

Also see #44589.