The package currently only supports storing the generated models as .jl files, which can then be included and precompiled in a package for estimation/etc. This is necessary for large models where model generation can be slow. For smaller models, there is no reason that the functions couldn't be done interactively - although Julia's "world age" compilation issues make it more difficult that it might first appear.
The downside of the current approach are that: (1) modules and julia files are difficult to work with interactively/overwrite/etc.; and (2) the current hack of loading everything in the module and then passing a module is easy on the frontend but is abusing the module system. The most egregious issue there is use of the module wrapper class and https://github.com/HighDimensionalEconLab/DifferentiableStateSpaceModels.jl/blob/v0.4.19/src/types.jl#L1-L13
We needed to add that hack because between julia versions the Module type no longer supported deepcopy, which broke multithreaded sampling in Turing because our perturbation model couldn't be copied.
The solution to all of these problems is to (1) move from passing a module to the PerturbationModel constuctor to passing in functions, and then support runtime generated functions for that construction - as opposed to only loading from a module.
To do a check to make sure you didn't miss anything, search in the sourcecode for .mod. and all of those should be in the problem type.
[ ] Change all cases of .mod.m. in the sourcecode and just change to . or something along those lines.
For example, in https://github.com/HighDimensionalEconLab/DifferentiableStateSpaceModels.jl/blob/v0.4.19/src/generate_perturbation.jl#L33 it should change from order_vector_by_symbols(merge(p_d, p_f), m.mod.m.p_symbols) to order_vector_by_symbols(merge(p_d, p_f), m.p_symbols)
The package currently only supports storing the generated models as
.jl
files, which can then be included and precompiled in a package for estimation/etc. This is necessary for large models where model generation can be slow. For smaller models, there is no reason that the functions couldn't be done interactively - although Julia's "world age" compilation issues make it more difficult that it might first appear.The downside of the current approach are that: (1) modules and julia files are difficult to work with interactively/overwrite/etc.; and (2) the current hack of loading everything in the module and then passing a module is easy on the frontend but is abusing the module system. The most egregious issue there is use of the module wrapper class and https://github.com/HighDimensionalEconLab/DifferentiableStateSpaceModels.jl/blob/v0.4.19/src/types.jl#L1-L13
We needed to add that hack because between julia versions the
Module
type no longer supported deepcopy, which broke multithreaded sampling in Turing because our perturbation model couldn't be copied.The solution to all of these problems is to (1) move from passing a module to the
PerturbationModel
constuctor to passing in functions, and then support runtime generated functions for that construction - as opposed to only loading from a module.To sketch out the steps:
mod
type in https://github.com/HighDimensionalEconLab/DifferentiableStateSpaceModels.jl/blob/v0.4.19/src/types.jl#L16-L31 and add in a huge number of generic type parameters for all of the underlying functions in the first and second order models. e.g.H!, H_yp!, H_x_p!, Ψ_yp!
etc. and all of the other things that are currently access from the.mod
type. For example,η, Q, u_symbols
etc..mod.
and all of those should be in the problem type..mod.m.
in the sourcecode and just change to.
or something along those lines.https://github.com/HighDimensionalEconLab/DifferentiableStateSpaceModels.jl/blob/v0.4.19/src/generate_perturbation.jl#L33
it should change fromorder_vector_by_symbols(merge(p_d, p_f), m.mod.m.p_symbols)
toorder_vector_by_symbols(merge(p_d, p_f), m.p_symbols)
m.mod.m.ȳ!(c.y, p)
tom.ȳ!(c.y, p)
function PerturbationModel(mod::Module)
and then drop themod
parameter and add all of those functions directly from the module itself.H!
as the last parameter then you could do something likeshow
in https://github.com/HighDimensionalEconLab/DifferentiableStateSpaceModels.jl/blob/v0.4.19/src/types.jl#L48-L51 to use the internal values and avoid the.mod.m.
.mod.
to make sure you didn't miss anything.Once that is done, we can add support for runtime functions in the next issue.