new forget() API

[staltz]

Context: defrag and compact, see https://github.com/ssb-ngi-pointer/ssb-db2/pull/339

Problem: if there is a feedId that we know we will never connect with again, we need a way to clean up the space it uses in disk for persistent EBT state. request() with false works for the runtime behavior, but it doesn't clean the state in disk. The state in disk can be significant, e.g. my Manyverse Desktop has a 96 MB large ebt folder.

Solution: new forget() API which just does request(false) but also deletes the state file.

[staltz]

Good point, I'll test that before merging.

[staltz]

Okay, I wrote a test which I pushed to this branch, and it fails (bob is supposed to have re-replicated alice, but he doesn't).

I don't know exactly how to solve this. It's in E-B-T for sure, but could have we have some way of Bob telling Alice to "force" replicate even if Alice thinks that Bob is up-to-date?

[staltz]

@arj03 I got to the bottom of this. It turns out the problem wasn't in E-B-T's state. E-B-T correctly re-requests the feed, and the messages are received again. It was that validate state was stuck, and there was this error happening silently:

Error: invalid message: expected sequence 2 but got: 1 in state:
{
  "id": "%Ei6QNg+J4f/9WWNIXd5op2EJ7JxGb7BBFsBACNrPJNo=.sha256",
  "sequence": 1,
  "timestamp": 1652964291658,
  "queue": []
}, on feed:@1b9KP8znF7A4i8wnSevBSK2ZabI/Re4bYF/Vh3hXasQ=.ed25519

Which makes a lot of sense, ssb-validate still thinks that the messages for a feed are there, so it doesn't accept appending older messages.

This was based on ssb-db and ssb-validate so I'm not sure yet if ssb-db2 is going to behave differently.

[staltz]

Yay the test passes if I replace ssb-db with ssb-db2! :tada:

[arj03]

Ah :) Makes sense. Good to see deletefeed working here