cockroachdb / cockroach

CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.
https://www.cockroachlabs.com
Other
30.08k stars 3.8k forks source link

pgwire: support wireprotocol compression #101631

Open dikshant opened 1 year ago

dikshant commented 1 year ago

PostgreSQL supports wire protocol compression. We should investigate if this is something that would provide any material benefit to us and if it is feasible to implement. https://www.postgresql.org/message-id/ABAA09C6-BB95-47A5-890D-90353533F9AC@yandex-team.ru

1. Compression can be requested by a client by including the "compression" option in its connection
string. This can either be a boolean value to enable or
disable compression or an explicit list of comma-separated compression algorithms which can
optionally include compression level. The client indicates the compression request by sending the
_pq_.compression startup packet
parameter with a list of compression algorithms and an optional specification of compression level. 
If the server does not support compression, the backend will ignore the _pq_.compression parameter
and will not send the CompressionAck message to the frontend. 

2. Server receives the client's compression request and intersects the requested compression
algorithms with the allowed ones (controlled via the libpq_compression server config setting). If
the intersection is not empty, the server responds with CompressionAck containing the final list of
the compression algorithms that can be used for the compression of libpq messages between the client
and server. If the intersection is empty (server does not accept any of the requested algorithms),
then it replies with CompressionAck containing the empty list and it is up to the client whether to
continue without compression or to report an error. 

3. After sending the CompressionAck message, the server can send the SetCompressionMethod message to
set the current compression algorithm for server-to-client traffic compression. Same for the client,
after receiving the CompressionAck message, the client can send the SetCompressionMethod message to set the current
compression algorithm for client-to-server traffic compression. Client-to-server and
server-to-client compression are independent of each other.

Jira issue: CRDB-27073

bdarnell commented 1 year ago

PostgreSQL supports wire protocol compression.

Does it? That linked mailing list thread is discussing work-in-progress that is being considered for postgres 16.

The proposed implementation looks a lot like TLS compression in that it encrypts the entire stream together. This raises concerns about vulnerabilities like CRIME. (especially if the authentication exchange is compressed - that would be a big problem in our original auth system although it's lessened with SCRAM).

Overall I'd bet that pgwire is much less exposed to compression-related leaks than HTTP (especially with scram), but we'd still need to evaluate any use of stream-wide compression from this perspective. (on the other hand, if we decide we're OK with stream compression, we may just be able to turn it on at the TLS level without waiting for future postgres protocol changes)

bdarnell commented 1 year ago

on the other hand, if we decide we're OK with stream compression, we may just be able to turn it on at the TLS level without waiting for future postgres protocol changes

Actually, TLS compression has fallen so completely out of favor after BREACH and CRIME that Go's TLS implementation doesn't support it at all.