thorek1 / MacroModelling.jl

Macros and functions to work with DSGE models.
https://thorek1.github.io/MacroModelling.jl/stable
MIT License
98 stars 15 forks source link

Performance claims #38

Closed gdalle closed 1 year ago

gdalle commented 1 year ago

I'm not sure if

The package stands out for its [...] effective steady state solver.

is a performance claim, but if it is, it needs to be backed up with benchmarks against competing packages.

If you're looking for inspiration, I recently did cross-language benchmarks (Julia/Python) for HiddenMarkovModels.jl. They are compatible with PkgBenchmark.jl and are also automatically added to the docs for each new release I tag.

https://github.com/openjournals/joss-reviews/issues/5598

thorek1 commented 1 year ago

usually the user derives the steady state conditions by hand or hopes that a numerical solver finds it automatically (which is far from being guaranteed)

what MacroModelling does different is that it used symbolic manipulation to reduce the steady state problem into smaller separate subproblems and then it uses effective algorithms (as in they find it, not they find it faster than other algorithms) to find the steady state. the algorithm uses a range of tricks, from search space transformation, automatic domain restrictions, restarts with different initial values, warm starts using previous solutions. these are not new in isolation but in this case they work very well in combination. by that I mean that they solve all 14 models implemented with MacroModelling out of the box without any starting point guess

The solver is a paper on it's own I think. Anyway, you think some of this info should be included?

@benchmarking: that's a whole other dimension I don't focus on here yet. It will become relevant at later stages of package development

gdalle commented 1 year ago

by that I mean that they solve all 14 models implemented with MacroModelling out of the box without any starting point guess

This is a good thing to include in the comparison if alternative packages cannot do that

thorek1 commented 1 year ago

added a few sentences to highlight why this feature is important, better than with other packages, and how the package achieves it

gdalle commented 1 year ago

perfect