Unknown loading error Dict of model, loss, params from Flux

[luboshanus]

Hi,

I have a problem that on server julia v1.2 I save oos predictions into a .bson. When I load those at this computer I successfully do. If I download them to my PC (Mac) julia v1.3.1 I get error see below. However, if I save, download, and load test.bson containing basic Dict() with randn(10,10) load works.

The file contains Dict{Symbol, Any} with 4 entries: ... and looks like this:

julia> aa = BSON.load("output/tt/asset_model_1_oos.bson")
Dict{Symbol,Any} with 4 entries:
  :a_oos_res     => Any[Any[[2952], 0.00335798, Float32[0.121619; 0.191023; … ;…
  :a_model       => Chain(Dense(11, 128, relu), Dense(128, 256, relu), Dense(25…
  :asset         => Any[1, :aapl, [-0.010887, 0.0324042, -0.0108924, 0.0177172,…
  :a_params_dict => Dict{Symbol,Any}(:loss=>0.000304821,:pl=>0.0009687,:lr=>0.0…

But when I download it to my computer and try to load it I get the error. I don't understand this behaviour and it is really frustraiting. I do not get the point why it works on one machine and not on the other one with this particular file and it does work with different new file on both.

BSON.load("output/tt/asset_model_1_oos.bson")
ERROR: InexactError: check_top_bit(Int64, -1)
Stacktrace:
 [1] throw_inexacterror(::Symbol, ::Type{Int64}, ::Int64) at ./boot.jl:560
 [2] check_top_bit at ./boot.jl:574 [inlined]
 [3] toUInt64 at ./boot.jl:685 [inlined]
 [4] UInt64 at ./boot.jl:715 [inlined]
 [5] convert(::Type{UInt64}, ::Int64) at ./number.jl:7
 [6] newstruct!(::Core.CodeInfo, ::Array{Any,1}, ::Array{Int32,1}, ::Int64, ::Array{UInt8,1}, ::Nothing, ::Array{Any,1}, ::Array{Any,1}, ::Array{UInt8,1}, ::Nothing, ::Type, ::Nothing, ::Int64, ::Vararg{Any,N} where N) at /Users/luboshanus/.julia/packages/BSON/lY9i8/src/extensions.jl:98
 [7] newstruct_raw(::IdDict{Any,Any}, ::Type, ::Dict{Symbol,Any}) at /Users/luboshanus/.julia/packages/BSON/lY9i8/src/extensions.jl:125
 [8] (::BSON.var"#45#46")(::Dict{Symbol,Any}, ::IdDict{Any,Any}) at /Users/luboshanus/.julia/packages/BSON/lY9i8/src/extensions.jl:140
 [9] raise_recursive(::Dict{Symbol,Any}, ::IdDict{Any,Any}) at /Users/luboshanus/.julia/packages/BSON/lY9i8/src/read.jl:88
 [10] (::BSON.var"#41#42"{IdDict{Any,Any}})(::Dict{Symbol,Any}) at /Users/luboshanus/.julia/packages/BSON/lY9i8/src/extensions.jl:124
 [11] collect_to!(::Array{Any,1}, ::Base.Generator{Array{Any,1},BSON.var"#41#42"{IdDict{Any,Any}}}, ::Int64, ::Int64) at ./generator.jl:47
 [12] collect_to!(::Array{Module,1}, ::Base.Generator{Array{Any,1},BSON.var"#41#42"{IdDict{Any,Any}}}, ::Int64, ::Int64) at ./array.jl:675
 [13] collect_to_with_first!(::Array{Module,1}, ::Module, ::Base.Generator{Array{Any,1},BSON.var"#41#42"{IdDict{Any,Any}}}, ::Int64) at ./array.jl:646
 [14] _collect(::Array{Any,1}, ::Base.Generator{Array{Any,1},BSON.var"#41#42"{IdDict{Any,Any}}}, ::Base.EltypeUnknown, ::Base.HasShape{1}) at ./array.jl:640
 [15] collect_similar(::Array{Any,1}, ::Base.Generator{Array{Any,1},BSON.var"#41#42"{IdDict{Any,Any}}}) at ./array.jl:564
 [16] map(::Function, ::Array{Any,1}) at ./abstractarray.jl:2073
 [17] newstruct_raw(::IdDict{Any,Any}, ::Type, ::Dict{Symbol,Any}) at /Users/luboshanus/.julia/packages/BSON/lY9i8/src/extensions.jl:124
 [18] (::BSON.var"#45#46")(::Dict{Symbol,Any}, ::IdDict{Any,Any}) at /Users/luboshanus/.julia/packages/BSON/lY9i8/src/extensions.jl:140
 [19] raise_recursive(::Dict{Symbol,Any}, ::IdDict{Any,Any}) at /Users/luboshanus/.julia/packages/BSON/lY9i8/src/read.jl:88
 [20] (::BSON.var"#23#24"{IdDict{Any,Any}})(::Dict{Symbol,Any}) at /Users/luboshanus/.julia/packages/BSON/lY9i8/src/read.jl:94
 [21] applychildren!(::BSON.var"#23#24"{IdDict{Any,Any}}, ::Array{Any,1}) at /Users/luboshanus/.julia/packages/BSON/lY9i8/src/BSON.jl:28
 [22] raise_recursive at /Users/luboshanus/.julia/packages/BSON/lY9i8/src/read.jl:94 [inlined]
 [23] (::BSON.var"#23#24"{IdDict{Any,Any}})(::Array{Any,1}) at /Users/luboshanus/.julia/packages/BSON/lY9i8/src/read.jl:94
 [24] applychildren!(::BSON.var"#23#24"{IdDict{Any,Any}}, ::Array{Any,1}) at /Users/luboshanus/.julia/packages/BSON/lY9i8/src/BSON.jl:28
 [25] raise_recursive(::Array{Any,1}, ::IdDict{Any,Any}) at /Users/luboshanus/.julia/packages/BSON/lY9i8/src/read.jl:94
 [26] newstruct_raw(::IdDict{Any,Any}, ::Type{Core.TypeName}, ::Dict{Symbol,Any}) at /Users/luboshanus/.julia/packages/BSON/lY9i8/src/anonymous.jl:71
 [27] (::BSON.var"#45#46")(::Dict{Symbol,Any}, ::IdDict{Any,Any}) at /Users/luboshanus/.julia/packages/BSON/lY9i8/src/extensions.jl:140
 [28] raise_recursive(::Dict{Symbol,Any}, ::IdDict{Any,Any}) at /Users/luboshanus/.julia/packages/BSON/lY9i8/src/read.jl:88
 [29] (::BSON.var"#19#21"{IdDict{Any,Any}})(::Dict{Symbol,Any}) at /Users/luboshanus/.julia/packages/BSON/lY9i8/src/read.jl:79
 [30] applychildren!(::BSON.var"#19#21"{IdDict{Any,Any}}, ::Dict{Symbol,Any}) at /Users/luboshanus/.julia/packages/BSON/lY9i8/src/BSON.jl:21
 [31] _raise_recursive(::Dict{Symbol,Any}, ::IdDict{Any,Any}) at /Users/luboshanus/.julia/packages/BSON/lY9i8/src/read.jl:79
 [32] raise_recursive(::Dict{Symbol,Any}, ::IdDict{Any,Any}) at /Users/luboshanus/.julia/packages/BSON/lY9i8/src/read.jl:89
 [33] (::BSON.var"#23#24"{IdDict{Any,Any}})(::Dict{Symbol,Any}) at /Users/luboshanus/.julia/packages/BSON/lY9i8/src/read.jl:94
 [34] applychildren!(::BSON.var"#23#24"{IdDict{Any,Any}}, ::Array{Any,1}) at /Users/luboshanus/.julia/packages/BSON/lY9i8/src/BSON.jl:28
 [35] raise_recursive at /Users/luboshanus/.julia/packages/BSON/lY9i8/src/read.jl:94 [inlined]
 ... (the last 7 lines are repeated 1 more time)
 [43] (::BSON.var"#19#21"{IdDict{Any,Any}})(::Array{Any,1}) at /Users/luboshanus/.julia/packages/BSON/lY9i8/src/read.jl:79
 [44] applychildren!(::BSON.var"#19#21"{IdDict{Any,Any}}, ::Dict{Symbol,Any}) at /Users/luboshanus/.julia/packages/BSON/lY9i8/src/BSON.jl:21
 [45] _raise_recursive(::Dict{Symbol,Any}, ::IdDict{Any,Any}) at /Users/luboshanus/.julia/packages/BSON/lY9i8/src/read.jl:79
 [46] raise_recursive(::Dict{Symbol,Any}, ::IdDict{Any,Any}) at /Users/luboshanus/.julia/packages/BSON/lY9i8/src/read.jl:89
 [47] (::BSON.var"#45#46")(::Dict{Symbol,Any}, ::IdDict{Any,Any}) at /Users/luboshanus/.julia/packages/BSON/lY9i8/src/extensions.jl:138
 [48] raise_recursive(::Dict{Symbol,Any}, ::IdDict{Any,Any}) at /Users/luboshanus/.julia/packages/BSON/lY9i8/src/read.jl:88
 [49] (::BSON.var"#20#22"{IdDict{Any,Any}})(::Dict{Symbol,Any}) at /Users/luboshanus/.julia/packages/BSON/lY9i8/src/read.jl:82
 [50] applychildren!(::BSON.var"#20#22"{IdDict{Any,Any}}, ::Dict{Symbol,Any}) at /Users/luboshanus/.julia/packages/BSON/lY9i8/src/BSON.jl:21
 [51] _raise_recursive(::Dict{Symbol,Any}, ::IdDict{Any,Any}) at /Users/luboshanus/.julia/packages/BSON/lY9i8/src/read.jl:82
 [52] raise_recursive(::Dict{Symbol,Any}, ::IdDict{Any,Any}) at /Users/luboshanus/.julia/packages/BSON/lY9i8/src/read.jl:89
 ... (the last 4 lines are repeated 1 more time)
 [57] raise_recursive at /Users/luboshanus/.julia/packages/BSON/lY9i8/src/read.jl:99 [inlined]
 [58] load(::String) at /Users/luboshanus/.julia/packages/BSON/lY9i8/src/read.jl:104

[bhvieira]

I would first ensure I'm using exactly the same version of the packages involved in both machines. Pkg.instantiate is great for this.

[luboshanus]

Thanks for the message. I tried that (having Pkg and then instantiate it with given dependencies) and I get the same error message as in previous.

BUT I have just installed julia v1.2.0 and I am able to open the .bson created at different machines with this version. And this is super unfortunate behaviour.

BSON is great but why it is not stable even my environment is the same [except julia versions 1.2 and 1.3. Could @MikeInnes help? Thank you!

(Sometimes it just looks like there are random magic bugs (in julia) complicating the work for fun or what...)

[racinmat]

Oh, so change of Julia between 1.2 and 1.3 is sometimes breaking for BSON? Good to know. Personally I'm migrating between Julia 1.3, 1.4 and 1.5 and luckily there the BSON deserialization works for me even for Flux models.