blowfishxyz / blocklist

Javascript library to use Blowfish blocklists locally
https://blowfish.xyz/
MIT License
11 stars 0 forks source link

feat: move to base64 for bitvector encoding (NIP-1111) #6

Closed sevazhidkov closed 1 year ago

sevazhidkov commented 1 year ago

After looking into encoded bloom filter sizes, I've decided that it would be better long-term to move to base64 for encoding, instead of encoding bits as byte-numbers. While it looks more obscure when inspecting JSON (that's why I initially rejected the idea of doing this), it delivers significant improvements on usage of local storage.

Bloom filter JSON size using []number: 1.8MB (672KB gzipped). Using base64 string: 719 KB (531KB gzipped).

Cloudfront automatically gzips all data, so this won't bring proper network improvements, but it would allow integrators to use much less data when storing the object in local storage (5MB limit per extension).

Tests are failed expectedly, since we haven't updated S3 bloom filter yet. All non-integration tests work properly.