mirage / irmin

Irmin is a distributed database that follows the same design principles as Git
https://irmin.org
ISC License
1.84k stars 153 forks source link

irmin-pack: clear LRU when reloading RO #2200

Closed metanivek closed 1 year ago

metanivek commented 1 year ago

Currently, when a RO instance calls reload, the LRU is not cleared. We have seen some issues in octez that we think is caused by this behavior. See #2171.

When opening that issue, I originally made the assumption we did not have what we needed to "smartly" clear the LRU for RO instances, but I had an idea upon second consideration. During reload, we only reopen the prefix if the generation changes so we can use that as our signal to clear the LRU. With some refactoring, that is what this PR does.

In order to clear the LRU when reload is called, this is a general refactor to move clearing of the LRU to happen when the prefix is reopened. This covers both a swap after a GC and when reload is called for a RO instance.

I added some new tests to verify the LRU has cleared for both a GC and a reload (and importantly it is not cleared if a GC has not occurred). They are a bit indirect by testing hits to the LRU, but get the job done. Open to other suggestions though!


This PR is to address #2171, but we shouldn't close that issue until this fix is merged to main. The reason this PR is targeted at 3.5 is that we will likely release this as a 3.5.2 release (for downstream octez usage), then we will forward port to main. It is unclear if we will also need to do a 3.6.1 release.