This means that all Microsimulation runs provide caching on all variables, and theoretically, some Simulation runs without datasets (e.g., household calculator) might also cache; this is as opposed to select variables being cached only when Microsimulation runs provide an exhaustive_parameter_deps value, as I believe was the method before.
I believe that when merging the changes that created
SimulationMacroCache
(https://github.com/PolicyEngine/policyengine-core/pull/224), we introduced a subtle change in logic that now applies caching too liberally.Prior to the change, there was a function within
Simulation
called_get_macro_cache
that did two things: determine if caching should be applied when calculating a variable and provide the cache path where necessary. The first check this method did is available at the line below: https://github.com/PolicyEngine/policyengine-core/blob/62e6be0d918f32e8bad85d85181223f2f50bad7e/policyengine_core/simulations/simulation.py#L1407 Basically, if a dataset was not provided, no caching was utilized; otherwise, all other checks occurred.SimulationMacroCache
broke_get_macro_cache
into two different methods, and the newcheck_macro_cache
method does a couple of checks around datasets, before this line: https://github.com/PolicyEngine/policyengine-core/blob/96a467aeb917f9771fd5ec28b227b1b61749f9ad/policyengine_core/simulations/simulation.py#L1435 So that if a dataset is provided, caching is definitely utilized, otherwise, all other checks occur.This means that all Microsimulation runs provide caching on all variables, and theoretically, some Simulation runs without datasets (e.g., household calculator) might also cache; this is as opposed to select variables being cached only when Microsimulation runs provide an
exhaustive_parameter_deps
value, as I believe was the method before.