Reading data file fails when original package that created the data changed

[davorh]

I have stored a large solution (4D array size ~ 4GB) from DifferentialEquations package. Meanwhile, I upgraded Julia and reinstalled the packages. Now when I try to read in a file with jldopen and try to access object named as original variable "solvec" I get this error shown below. Is there any way to access my data?

┌ Warning: read type ODEFunction{false,SciMLBase.AutoSpecialize,Main.#BK,LinearAlgebra.UniformScaling{Bool},Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,typeof(SciMLBase.DEFAULT_OBSERVED),Nothing,Nothing} has a different number of parameters from type ODEFunction in workspace; reconstructing
└ @ JLD2 C:\Users\davorh\.julia\packages\JLD2\r5t7Q\src\data\reconstructing_datatypes.jl:457

┌ Warning: read type ODESolution{Float64, 3, Vector{Matrix{Float64}}, Nothing, Nothing, Vector{Float64}, Vector{Vector{Matrix{Float64}}}, ODEProblem{Matrix{Float64}, Tuple{Float64, Float64}, false, SciMLBase.NullParameters, JLD2.ReconstructedTypes.var"##ODEFunction{false,SciMLBase.AutoSpecialize,Main.#BK,LinearAlgebra.UniformScaling{Bool},Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,typeof(SciMLBase.DEFAULT_OBSERVED),Nothing,Nothing}#291", Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}, SciMLBase.StandardODEProblem}, AB3, OrdinaryDiffEq.InterpolationData{JLD2.ReconstructedTypes.var"##ODEFunction{false,SciMLBase.AutoSpecialize,Main.#BK,LinearAlgebra.UniformScaling{Bool},Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,typeof(SciMLBase.DEFAULT_OBSERVED),Nothing,Nothing}#291", Vector{Matrix{Float64}}, Vector{Float64}, Vector{Vector{Matrix{Float64}}}, OrdinaryDiffEq.AB3ConstantCache{Matrix{Float64}}}, DiffEqBase.DEStats} is not a leaf type in workspace; reconstructing
└ @ JLD2 C:\Users\davorh\.julia\packages\JLD2\r5t7Q\src\data\reconstructing_datatypes.jl:263

MethodError: Cannot `convert` an object of type 
  JLD2.ReconstructedTypes.var"##ODESolution{Float64, 3, Vector{Matrix{Float64}}, Nothing, Nothing, Vector{Float64}, Vector{Vector{Matrix{Float64}}}, ODEProblem{Matrix{Float64}, Tuple{Float64, Float64}, false, SciMLBase.NullParameters, JLD2.ReconstructedTypes.var\"##ODEFunction{false,SciMLBase.AutoSpecialize,Main.#BK,LinearAlgebra.UniformScaling{Bool},Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,typeof(SciMLBase.DEFAULT_OBSERVED),Nothing,Nothing}#291\", Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}, SciMLBase.StandardODEProblem}, AB3, OrdinaryDiffEq.InterpolationData{JLD2.ReconstructedTypes.var\"##ODEFunction{false,SciMLBase.AutoSpecialize,Main.#BK,LinearAlgebra.UniformScaling{Bool},Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,typeof(SciMLBase.DEFAULT_OBSERVED),Nothing,Nothing}#291\", Vector{Matrix{Float64}}, Vector{Float64}, Vector{Vector{Matrix{Float64}}}, OrdinaryDiffEq.AB3ConstantCache{Matrix{Float64}}}, DiffEqBase.DEStats}#292" to an object of type 
  ODESolution{Float64, 3, Vector{Matrix{Float64}}, Nothing, Nothing, Vector{Float64}, Vector{Vector{Matrix{Float64}}}, ODEProblem{Matrix{Float64}, Tuple{Float64, Float64}, false, SciMLBase.NullParameters, JLD2.ReconstructedTypes.var"##ODEFunction{false,SciMLBase.AutoSpecialize,Main.#BK,LinearAlgebra.UniformScaling{Bool},Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,typeof(SciMLBase.DEFAULT_OBSERVED),Nothing,Nothing}#291", Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}, SciMLBase.StandardODEProblem}, AB3, OrdinaryDiffEq.InterpolationData{JLD2.ReconstructedTypes.var"##ODEFunction{false,SciMLBase.AutoSpecialize,Main.#BK,LinearAlgebra.UniformScaling{Bool},Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,typeof(SciMLBase.DEFAULT_OBSERVED),Nothing,Nothing}#291", Vector{Matrix{Float64}}, Vector{Float64}, Vector{Vector{Matrix{Float64}}}, OrdinaryDiffEq.AB3ConstantCache{Matrix{Float64}}}, DiffEqBase.DEStats}

[JonasIsensee]

It looks like you are trying to load data with a new version of DifferentialEquations.jl that is not compatible with the older version. You can try to find out which version you used when writing the file and install that (older) version to load your data.

[davorh]

This part is perfectly clear, but it is a dependency hell. Is there any other option to save the data?

[JonasIsensee]

You could think about what data you really need to store. It is probably not the full DifferentialEquations.jl solution object but rather the result arrays directly. That is a bit of extra work but much more reliable.

[davorh]

This part is also perfectly clear. And we will do that in the future by avoiding JLD2, i.e. using more robust data storage and direct arrays.

[JonasIsensee]

Then I don't understand your question. There is no magic solution that would tell JLD2 to guess what your data means and what should be done to generate new changed structures from this. If you know a method to upgrade the old structure to the new one you can use the steps outlined in the documentation: https://juliaio.github.io/JLD2.jl/dev/advanced/