Some events that we were sending out over the PDS firehose were missing blocks necessary to validate the included proofs. This seems to have been primarily happening on events that contained a single delete, but hypothetically could happen on any write.
It seems like this was being caused by a reorganization of some blocks in the tree. Such that a block that was in the proof chain for delete in question existing previously in the tree. The differ would not return it as a new block in that case which is strictly true but not sufficient for a non-fully-authorized relay consumer.
This adds in the notion of "relevant blocks" to commit data as well. This include exactly the blocks that make up the proofs for the included operations.
This is an overlapping set with new blocks (often the same set). But "new blocks" can contain blocks not necessary for the included proof & similarly relevant blocks can include already existing blocks that are relevant to the proof.
When sending an event from the PDS, we send the union of these two block maps.
Some events that we were sending out over the PDS firehose were missing blocks necessary to validate the included proofs. This seems to have been primarily happening on events that contained a single delete, but hypothetically could happen on any write.
It seems like this was being caused by a reorganization of some blocks in the tree. Such that a block that was in the proof chain for delete in question existing previously in the tree. The differ would not return it as a new block in that case which is strictly true but not sufficient for a non-fully-authorized relay consumer.
This adds in the notion of "relevant blocks" to commit data as well. This include exactly the blocks that make up the proofs for the included operations.
This is an overlapping set with new blocks (often the same set). But "new blocks" can contain blocks not necessary for the included proof & similarly relevant blocks can include already existing blocks that are relevant to the proof.
When sending an event from the PDS, we send the union of these two block maps.