d-Rickyy-b / certstream-server-go

This project aims to be a drop-in replacement for the certstream server by Calidog. This tool aggregates, parses, and streams certificate data from multiple certificate transparency logs via websocket connections to the clients.
MIT License
82 stars 9 forks source link

Ability to send certs compressed so less bandwidth is used #30

Open d-Rickyy-b opened 7 months ago

d-Rickyy-b commented 7 months ago

Currently each "lite" cert is around 1500 bytes in size. That results in ~15 Mbit/s of data being sent over the network per client.

By compressing the data, we should be able to reduce that. An example cert of size 1449 bytes compressed via gzip resulted in 805 bytes. That makes a compression ratio of 1.8 and hence saves about 44.4% of data, resulting in ~8.25 Mbit/s per client.

Raw deflate of the same example cert resulted in a compression to 787 bytes. Making a compression ratio of 1.84 and saves about 45.7% of data, resulting in ~8.15 Mbit/s

Brotli takes us down to 710 bytes. Compression ration: 2.04 and saves about 51% of data, resulting in 7.39 Mbit/s


Still: Adding compression means that there is more workload on the clients to be done, which also means that these could potentially process certs even slower than before. Some testing is necessary before implementing compression into certstream.