JuliaIO / BSON.jl

Other
158 stars 39 forks source link

Name of local custom types not resolved #117

Open jmarbet opened 1 year ago

jmarbet commented 1 year ago

91 fixed an issue related to resolving the names of custom types. However, this only seems to work when the custom types are defined within separate packages, as in the case of the example in the README file.

Consider the following script that generates data.bson

using BSON
include("BModule.jl")
using .BModule

x = BStruct(1)
bson("data.bson", Dict(:x=>x))

with BModule.jl being

module BModule

    export BStruct

    struct BStruct
        b
    end

end

If I start a new Julia session and want to read data.bson from a module (without running the code above), e.g., I only run the following in REPL

module CModule

using BSON
include("BModule.jl")
using .BModule

x = BSON.load("data.bson", @__MODULE__)

end

I get an error message that BModule is not defined

ERROR: UndefVarError: BModule not defined
Stacktrace:
  [1] #31
    @ ~/.julia/packages/BSON/73cTU/src/extensions.jl:21 [inlined]
  [2] (::Base.BottomRF{BSON.var"#31#32"})(acc::Module, x::String)
    @ Base ./reduce.jl:81
  [3] _foldl_impl(op::Base.BottomRF{BSON.var"#31#32"}, init::Module, itr::Vector{Any})
    @ Base ./reduce.jl:62
  [4] foldl_impl
    @ ./reduce.jl:48 [inlined]
  [5] mapfoldl_impl
    @ ./reduce.jl:44 [inlined]
  [6] _mapreduce_dim
    @ ./reducedim.jl:362 [inlined]
  [7] #mapreduce#765
    @ ./reducedim.jl:357 [inlined]
  [8] #reduce#767
    @ ./reducedim.jl:406 [inlined]
  [9] resolve(fs::Vector{Any}, init::Module)
    @ BSON ~/.julia/packages/BSON/73cTU/src/extensions.jl:21
 [10] (::BSON.var"#35#36")(d::Dict{Symbol, Any}, init::Module)
    @ BSON ~/.julia/packages/BSON/73cTU/src/extensions.jl:64
 [11] _raise_recursive(d::Dict{Symbol, Any}, cache::IdDict{Any, Any}, init::Module)
    @ BSON ~/.julia/packages/BSON/73cTU/src/read.jl:80
 [12] raise_recursive(d::Dict{Symbol, Any}, cache::IdDict{Any, Any}, init::Module)
    @ BSON ~/.julia/packages/BSON/73cTU/src/read.jl:93
 [13] (::BSON.var"#49#50")(d::Dict{Symbol, Any}, cache::IdDict{Any, Any}, init::Module)
    @ BSON ~/.julia/packages/BSON/73cTU/src/extensions.jl:167
 [14] raise_recursive(d::Dict{Symbol, Any}, cache::IdDict{Any, Any}, init::Module)
    @ BSON ~/.julia/packages/BSON/73cTU/src/read.jl:92
 [15] (::BSON.var"#19#22"{IdDict{Any, Any}, Module})(x::Dict{Symbol, Any})
    @ BSON ~/.julia/packages/BSON/73cTU/src/read.jl:86
 [16] applychildren!(f::BSON.var"#19#22"{IdDict{Any, Any}, Module}, x::Dict{Symbol, Any})
    @ BSON ~/.julia/packages/BSON/73cTU/src/BSON.jl:19
 [17] _raise_recursive(d::Dict{Symbol, Any}, cache::IdDict{Any, Any}, init::Module)
    @ BSON ~/.julia/packages/BSON/73cTU/src/read.jl:86
 [18] raise_recursive(d::Dict{Symbol, Any}, cache::IdDict{Any, Any}, init::Module)
    @ BSON ~/.julia/packages/BSON/73cTU/src/read.jl:93
 [19] raise_recursive
    @ ~/.julia/packages/BSON/73cTU/src/read.jl:103 [inlined]
 [20] load(x::String, init::Module)
    @ BSON ~/.julia/packages/BSON/73cTU/src/read.jl:108
 [21] top-level scope
    @ REPL[1]:7

The issue seems to be related to the fact that Main is part of the name of the custom type (Main.BModule.BStruct in this case) for locally defined custom types, but it is not part of the names of types defined within packages. #91 does not seem to take that into account.

For my example above, I found two potential workarounds: 1. load BModule in Main from where I have generated data.bson, or 2. redefine resolve() to make sure that the function is looking in the correct namespace, e.g.,

module CModule

using BSON
include("BModule.jl")
using .BModule

function BSON.resolve(fs, init) 

    if first(fs) == "Main"
        fs = fs[2:end]
    end

    reduce((m, f) -> getfield(m, Symbol(f)), fs; init = init)

end

x = BSON.load("data.bson", @__MODULE__)

end

Solution 1 does not really fix the problem and can become cumbersome if the BSON file was also generated within a module and not in Main. Solution 2 is a bit of a hack, but I hope it provides a starting point for people more knowledgeable than me.