When we send messages repeatedly to the same peer we spend most of our time base64 encoding data to dedupe messages.
This can happen when sending data to Kubo since it only sends us max 32x wants in one message, so for a large DAG we can receive many messages in quick succession.
Instead, encode the keys for blocks/wants/presences in base64 at message creation time, then when we need to merge messages it's a simple string comparison instead.
We need strings for comparison because JavaScript's Map type uses === to compare key values, if there was a Comparator-style interface we'd be able to use Uint8Arrays for keys, but there isn't, so we can't.
Change checklist
[x] I have performed a self-review of my own code
[ ] I have made corresponding changes to the documentation if necessary (this includes comments as well)
[x] I have added tests that prove my fix is effective or that my feature works
When we send messages repeatedly to the same peer we spend most of our time
base64encoding data to dedupe messages.This can happen when sending data to Kubo since it only sends us max 32x wants in one message, so for a large DAG we can receive many messages in quick succession.
Instead, encode the keys for blocks/wants/presences in base64 at message creation time, then when we need to merge messages it's a simple string comparison instead.
We need strings for comparison because JavaScript's
Maptype uses===to compare key values, if there was aComparator-style interface we'd be able to useUint8Arrays for keys, but there isn't, so we can't.Change checklist