The per-thread `empty!()` call in `compile()` is broken on julia master.

[NHDaly]

Julia master recently merged https://github.com/JuliaLang/julia/pull/49094, which moved the interactive threadpool to come before the default threadpool.

This means that the threadids you get from iterating @threads :static no longer match up with the indexes. For example, here is julia started with JULIA_NUM_THREADS=4,1 julia --proj:

julia> VERSION
v"1.10.0-DEV.1118"

julia> Threads.nthreads()
4

julia> Threads.maxthreadid()
5

julia> Threads.@threads :static for i in 1:Threads.nthreads()
           @info i, Threads.threadid()
           @assert Threads.threadid() === i "TimeZones.TZData.compile() must be called from the main, top-level Task."
       end
[ Info: (1, 2)
[ Info: (3, 4)
[ Info: (4, 5)
[ Info: (2, 3)
ERROR: TaskFailedException

    nested task error: AssertionError: TimeZones.TZData.compile() must be called from the main, top-level Task.
    Stacktrace:
     [1] macro expansion
       @ ./REPL[5]:3 [inlined]
     [2] (::var"#18#threadsfor_fun#4"{var"#18#threadsfor_fun#3#5"{UnitRange{Int64}}})(tid::Int64; onethread::Bool)
       @ Main ./threadingconstructs.jl:163
     [3] #18#threadsfor_fun
       @ ./threadingconstructs.jl:130 [inlined]
     [4] (::Base.Threads.var"#1#2"{var"#18#threadsfor_fun#4"{var"#18#threadsfor_fun#3#5"{UnitRange{Int64}}}, Int64})()
       @ Base.Threads ./threadingconstructs.jl:108

...and 3 more exceptions.

Using for i in 1:Threads.maxthreadid() doesn't work either, because it only iterates in the current thread pool:

julia> Threads.@threads :static for i in 1:Threads.maxthreadid()
           @info i, Threads.threadid()
       end
[ Info: (4, 4)
[ Info: (5, 5)
[ Info: (1, 2)
[ Info: (3, 3)
[ Info: (2, 2)

(note there are two iterations with threadid() = 2.)

.........

It doesn't seem like there's any way to run a Threads.@threads over the interactive threads, from what I can see. Not 100% sure about that.

---

It might be that we need to either change the way this works so that instead of eagerly emptying all the caches, we do something more cooperative, like setting some "time to reset" bit for each thread, which is lazily checked on the first access on each thread and the cache is cleared before doing anything else? Or maybe triggering a channel that each thread has a task waiting on to clear its cache.

Or maybe we need to make a change in Julia to provide another possible schedule to @threads, like :interactive to force it to schedule over the interactive threadpool? :/

CC: @omus, @jeffbezanson

[NHDaly]

Oh, i forgot to link the broken code. it's here: https://github.com/JuliaTime/TimeZones.jl/blame/master/src/tzdata/compile.jl#L700-L703

And it breaks for us in our Pkg.build() step, because we have JULIA_NUM_THREADS=4,1 set for our entire CI job.

    Building TimeZones ───────────────→ `~/.julia/scratchspaces/44cfe95a-1eb2-52ea-b672-e2afdf69b78f/a92ec4466fc6e3dd704e2668b5e7f24add36d242/build.log`
ERROR: LoadError: Error building `TimeZones`: 
[ Info: Installing 2022f tzdata region data
[ Info: Converting tz source files into TimeZone data
ERROR: LoadError: TaskFailedException

    nested task error: AssertionError: TimeZones.TZData.compile() must be called from the main, top-level Task.
    Stacktrace:
     [1] macro expansion
       @ ~/.julia/packages/TimeZones/V28u7/src/tzdata/compile.jl:701 [inlined]
     [2] (::TimeZones.TZData.var"#195#threadsfor_fun#34"{TimeZones.TZData.var"#195#threadsfor_fun#32#35"{UnitRange{Int64}}})(tid::Int64; onethread::Bool)
       @ TimeZones.TZData ./threadingconstructs.jl:163
     [3] #195#threadsfor_fun
       @ ./threadingconstructs.jl:130 [inlined]
     [4] (::Base.Threads.var"#1#2"{TimeZones.TZData.var"#195#threadsfor_fun#34"{TimeZones.TZData.var"#195#threadsfor_fun#32#35"{UnitRange{Int64}}}, Int64})()
       @ Base.Threads ./threadingconstructs.jl:108

...and 3 more exceptions.

Stacktrace:
  [1] threading_run(fun::TimeZones.TZData.var"#195#threadsfor_fun#34"{TimeZones.TZData.var"#195#threadsfor_fun#32#35"{UnitRange{Int64}}}, static::Bool)
    @ Base.Threads ./threadingconstructs.jl:120
  [2] macro expansion
    @ ./threadingconstructs.jl:172 [inlined]
  [3] compile(tz_source::TimeZones.TZData.TZSource, dest_dir::String; kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, @NamedTuple{}})
    @ TimeZones.TZData ~/.julia/packages/TimeZones/V28u7/src/tzdata/compile.jl:700
...

[DilumAluthge]

cc: @vtjnash

[vtjnash]

That looks like a bad data race. Use a mutex if you want sane behavior from shared data.

[omus]

I think we can probably side step this issue with #382. I'll see if I can get that in this week

[omus]

Fixed by #382