Closed ignoramous closed 1 year ago
Used xdelta with alexkasko/delta-updater and the result of diff'ing two blocklists was around 76M for just td
(trie-data).
java -jar ./delta-diff/target/delta-diff-1.1.4-SNAPSHOT-distr/delta-diff.jar ../serverless-dns/blocklists__/2022/1668635781244/u6 ../serverless-dns/blocklists__/2022/1669339013468/u6 -o patch.zip
So, the generic delta / diff updates won't rescue us here.
VcDiff wasn't any better either ~48MB for trie.
It doesn't looks like off-the-shelf delta / diff algorithms are going to work for us.
This requires some more thought and possibly a custom diff / delta algorithm, for which there's not enough planning or time, right now. May visit a similar approach in the future, if needed.
Update more frequently (once every hour?), and see if delta updates are possible? Ship only bytes that have changed since the previous version. Like a git-patch.
https://archive.is/RiUEl