Open brookslogan opened 2 years ago
We can be a lot more memory-friendly than an in-memory list/df of snapshots, by only reading in one / a few snapshots at a time, and compactifying along the way (plus suggesting to save the compactified version on disk if we compactified a lot, or more complicated things regarding proposed "updating" archives that are out of scope for this initial feature).
See #172 for background. This would work around some of the caveats noted there by:
next_after(version)
was provided. If not, insert a snapshot revising all observations to NA with this version tag (next_after(version)
). ---- This is pretty inefficient space-wise. Maybe we would want to do something with the design of epi_archive to make more efficient options available.