Currently there is (unsafe) ad hoc caching implemented for the KDE stage (PR #689).
We should make this safer, and on a more global level.
Ideas:
I think cleanest would be to implement stashing in general at the pipeline level.
So you would stay: stash at stage X, and then the pipeline will not run anything up to stage X, except when a parameter has changed.
It probably should then keep an internal copy of some keys and apply those, just to avoid side effects, but that is low cost.
we could make the user specify what keys to be cached! So this way would be explicit.
E.g. in the cfg specify: At stage X cache keys x, and y
We could do that in the [pipeline] section of the cfg most naturally.
Add a optional: stash = {5 : ['weights', 'blah', ...]} or so
Currently there is (unsafe) ad hoc caching implemented for the KDE stage (PR #689).
We should make this safer, and on a more global level.
Ideas:
I think cleanest would be to implement stashing in general at the pipeline level. So you would stay: stash at stage X, and then the pipeline will not run anything up to stage X, except when a parameter has changed. It probably should then keep an internal copy of some keys and apply those, just to avoid side effects, but that is low cost. we could make the user specify what keys to be cached! So this way would be explicit. E.g. in the cfg specify: At stage X cache keys x, and y We could do that in the [pipeline] section of the cfg most naturally. Add a optional: stash = {5 : ['weights', 'blah', ...]} or so