search

CliMA / EnsembleKalmanProcesses.jl

Implements Optimization and approximate uncertainty quantification algorithms, Ensemble Kalman Inversion, and Ensemble Kalman Processes.
https://clima.github.io/EnsembleKalmanProcesses.jl/dev
Apache License 2.0
85 stars 19 forks source link

Add this test workflow for more robust back-compatability #403

Open odunbar opened 2 months ago

odunbar commented 2 months ago

https://github.com/CliMA/ClimaAtmos.jl/pull/3276/files

"downgrade"

Julians42 commented 1 month ago

Maybe not the same issue but I'm having backwards compatability issues using EKP v2 with old eki objects. I'm getting the following issue:

julia> eki = load_object("../output/local_calibration/iteration_000/eki_file.jld2")
┌ Warning: saved type EnsembleKalmanProcess{Float64, Int64, Inversion, DataMisfitController{Float64, Matrix{Float64}, String}, DefaultAccelerator} is missing field error in workspace type; reconstructing
└ @ JLD2 ~/.julia/packages/JLD2/OP0XX/src/data/reconstructing_datatypes.jl:195
┌ Warning: type EnsembleKalmanProcesses.#failsafe_update#121 does not exist in workspace; reconstructing
└ @ JLD2 ~/.julia/packages/JLD2/OP0XX/src/data/reconstructing_datatypes.jl:489
ERROR: MethodError: Cannot `convert` an object of type JLD2.ReconstructedSingleton{Symbol("#failsafe_update#121")} to an object of type Function
odunbar commented 1 month ago

Hi @Julians42,

The issue I think you see is that you likely used an old JLD2 version to save a data object, then updated JLD2 (possibly to a new breaking version) and the new version can't read data saved by the old.

This is not quite the same, as EKP is compatible with both JLD2 versions (new and old). you can resolve your setup by pinning a JLD2 version. Or your could rerun experiments with the new JLD2. It is unfortunately a cost of using these types of save data. I'm not sure if there is some conversion yet. though it would be nice to have a loader with v1 and then be able to update it to save with v2