HighDimensionalEconLab / DifferentiableStateSpaceModels.jl

MIT License
46 stars 1 forks source link

Move from requiring modules to storing functions in problem structure #152

Open jlperla opened 2 years ago

jlperla commented 2 years ago

The package currently only supports storing the generated models as .jl files, which can then be included and precompiled in a package for estimation/etc. This is necessary for large models where model generation can be slow. For smaller models, there is no reason that the functions couldn't be done interactively - although Julia's "world age" compilation issues make it more difficult that it might first appear.

The downside of the current approach are that: (1) modules and julia files are difficult to work with interactively/overwrite/etc.; and (2) the current hack of loading everything in the module and then passing a module is easy on the frontend but is abusing the module system. The most egregious issue there is use of the module wrapper class and https://github.com/HighDimensionalEconLab/DifferentiableStateSpaceModels.jl/blob/v0.4.19/src/types.jl#L1-L13

We needed to add that hack because between julia versions the Module type no longer supported deepcopy, which broke multithreaded sampling in Turing because our perturbation model couldn't be copied.

The solution to all of these problems is to (1) move from passing a module to the PerturbationModel constuctor to passing in functions, and then support runtime generated functions for that construction - as opposed to only loading from a module.

To sketch out the steps:

Once that is done, we can add support for runtime functions in the next issue.