Open 70Gage70 opened 1 week ago
Hi @70Gage70,
thanks for the report. I ran your examples and can reproduce.
Since everything works on v1.10, and on v1.11 as well (as long as it is not executed during package loading)
I don't think this is really a JLD2 issue but more on the julia side.
I tested that without __init__
you can also load the data on first try even on v1.11.
Here's a further reduced problem: (remove Interpolations, make MyStruct
not be parametric and just store something like an Int)
julia> using ItpScratchJLD
┌ Warning: type ItpScratchJLD.MyStruct does not exist in workspace; reconstructing
So it seems that the module ItpScratchJLD
is no longer added to Main
before running __init__
@JonasIsensee thanks for your comments. I did a quick search on open Julia issues related to __init__
, it looks like my issue is possibly a subset of this one. I confirmed that pkgdir
prints nothing
on 1.11.1 but the correct directory on 1.10.5.
I have a similar problem. I load some data inside __init__
which contains structs defined in the package itself. This previously worked fine (e.g., on 1.10) but on v1.11, JLD2 cannot recognize the types - even though they are defined in the package the __init__
is being called from.
Can confirm that this works on Julia v1.11.0-rc1, but not on later versions - so seems very likely to be due to the above-linked Julia issue, cf. this comment https://github.com/JuliaLang/julia/issues/56077#issuecomment-2403088414.
I'm not sure if this is happening because of Scratch.jl, Interpolations.jl or JLD2.jl, but the error is raised by JLD2.jl so I'm posting my MWE here. See the following little package
src
```julia module ItpScratchJLD export BAR, create_itp, load_itp using Scratch, JLD2, Interpolations struct MyStruct{I} itp::I end PATH_SCRATCH = Ref{String}() BAR = Ref{MyStruct}() function create_itp(; path = PATH_SCRATCH.x) outfile = joinpath(path, "itp.jld2") rm(outfile, force = true) itp = cubic_spline_interpolation(1:10, 1:10) |> MyStruct jldsave(outfile, itp = itp) end function load_itp(; path = PATH_SCRATCH.x) BAR.x = load(joinpath(path, "itp.jld2"), "itp") end function __init__() PATH_SCRATCH.x = @get_scratch!("TEST") try load_itp() catch nothing end end end # module ```Project.toml
```toml name = "ItpScratchJLD" uuid = "f4526136-8e65-11ef-0aeb-f95001e491ea" authors = ["Test"] version = "0.0.1" [deps] Interpolations = "a98d9a8b-a2ab-59e6-89dd-64a1c18fca59" JLD2 = "033835bb-8acc-5ee8-8aae-3f567f8a3819" Scratch = "6c6a2e73-6563-6170-7368-637461726353" ```This (roughly) reproduces the workflow of a much larger package, where interpolants are created and saved to a scratchspace using JLD2.jl so that they can be loaded later without having to recompute everything. If the interpolants exist, they are loaded automatically by
__init__()
.On 1.10.5, if you
using ItpScratchJLD
, then runcreate_itp()
, then close everything, thenusing ItpScratchJLD
again, you find thatBAR
is set correctly.Now doing the same thing on 1.11.1, the warning is raised and
BAR
is not loaded, although it can be loaded afterwards with no issue.WARNING MESSAGE TEXT
```toml ┌ Warning: type ItpScratchJLD.MyStruct{Interpolations.Extrapolation{Float64, 1, Interpolations.ScaledInterpolation{Float64, 1, Interpolations.BSplineInterpolation{Float64, 1, OffsetArrays.OffsetVector{Float64, Vector{Float64}}, Interpolations.BSpline{Interpolations.Cubic{Interpolations.Line{Interpolations.OnGrid}}}, Tuple{Base.OneTo{Int64}}}, Interpolations.BSpline{Interpolations.Cubic{Interpolations.Line{Interpolations.OnGrid}}}, Tuple{UnitRange{Int64}}}, Interpolations.BSpline{Interpolations.Cubic{Interpolations.Line{Interpolations.OnGrid}}}, Interpolations.Throw{Nothing}}} does not exist in workspace; reconstructing └ @ JLD2 ~/.julia/packages/JLD2/IyyZz/src/data/reconstructing_datatypes.jl:577 ```Very tricky.