Currently, every pass with AbcSmc must ingest the entire history of complete jobs to process the latest set => propose next set. This is because the weights are computed iteratively: the latest set weights actually depend on the entire history of weights. We don't store those weights, nor the parameters for noising draws.
We should continue to store the history, but if we also start storing weights and doubled variances, then we can simplify the code that deals with filtering to just take the last complete step. This could allow us to substantially trim down the AbcSmc object itself, getting rid of all the vectors-of-stuff and the assorted look-up-by-set accessors.
At the moment, we have no fitting-algorithmic reason to use all the data, though we do look at it for diagnostic reasons. We don't generally need an AbcSmc object, however, to do the diagnostic steps.
Currently, every pass with AbcSmc must ingest the entire history of complete jobs to process the latest set => propose next set. This is because the weights are computed iteratively: the latest set weights actually depend on the entire history of weights. We don't store those weights, nor the parameters for noising draws.
We should continue to store the history, but if we also start storing weights and doubled variances, then we can simplify the code that deals with filtering to just take the last complete step. This could allow us to substantially trim down the AbcSmc object itself, getting rid of all the vectors-of-stuff and the assorted look-up-by-set accessors.
At the moment, we have no fitting-algorithmic reason to use all the data, though we do look at it for diagnostic reasons. We don't generally need an AbcSmc object, however, to do the diagnostic steps.