search

grobian / carbon-c-relay

Enhanced C implementation of Carbon relay, aggregator and rewriter
Apache License 2.0
380 stars 107 forks source link

Add support for transport gzip over ssl #323

Closed iain-buclaw-sociomantic closed 6 years ago

iain-buclaw-sociomantic commented 6 years ago

Currently, the relay config for listen and cluster only allows specifying one or the other, not both combined.

grobian commented 6 years ago

I knew this would be coming :) But SSL uses gzip underneath, so it would make little (actually no) sense to apply both.

iain-buclaw-sociomantic commented 6 years ago

Haha. Well when looking at tcpdump between two relays, the gzip packets were significantly smaller that the ssl (which in turn were significantly smaller than plain-text).

If SSL gzips, I wonder if its the difference between gzip+encrypt vs. encrypt+gzip. The latter may not compress as well as the former I guess.

iain-buclaw-sociomantic commented 6 years ago

Actually, I partially retract from my previous statement.

I'm just monitoring sending data between two carbon-c-relays using production load (67K metrics per second), and these are the network bandwidth figures I'm seeing.

Transport Bandwidth Packets
plain-text 4.1MB/s 2.8K/s
ssl 5.6MB/s 3.9K/s
gzip 325KB/s 239/s

I find it surprising that ssl adds that much overhead over plain-text, but I think it's pretty clear that this presents an argument for gzip+ssl.

grobian commented 6 years ago

https://stackoverflow.com/questions/2767211/can-you-use-gzip-over-ssl-and-connection-keep-alive-headers

maybe is interesting read here

grobian commented 6 years ago

FWIW I'm still puzzled, I would agree that gzip + ssl would make "sense", from your tests it shows clearly this isn't what's done now.

grobian commented 6 years ago

I gave this thing a spin, on "internet free moments" during my vacation, decided to push to ensure the work wouldn't get lost, but did minimal testing on this one.