cloudevents / spec

CloudEvents Specification
https://cloudevents.io
Apache License 2.0
5.05k stars 582 forks source link

Content Encoding Extension (i.e. compression) #1111

Closed alexec closed 1 year ago

alexec commented 2 years ago

Like encryption, we need a way to indicate that the data is compressed. In HTTP we would use Content-Encoding: gzip, but CloudEvents does not have this. Even if it did, we would not use it, because, unlike HTTP, the Content-Type (i.e. datacontenttype) is not contextualized with Content-Encoding. We need an extension to standardize both the encoding and the encoded type:

All context attributes are namespaced by cte.

{
  "specversion": "1.0",
  "source": "s",
  "id": "i-0",
  "type": "example.v1",
  "cte": "gzip",
  "ctecontenttype": "application/json",
  "datacontenttype": "application/octet-stream",
  "data_base64": "abc123="
}

When using Application Level Encryption, we cannot utilize the transport encryption. The data must be encrypted. To reduce message size, we must encrypt the data, but compression after encryption does not reduce messages size (because encrypted data, by definition, has so much entropy it cannot be compressed).

I'd classify both encryption and compression as data transformations that can be stacked.

data=encrypt(compress(data))

Stacking is ordered, so that we might need a way to indicate the order of the stacking. Or we might not, e.g. order is forced by the binding.

clemensv commented 1 year ago

We had that and dropped it again.

Added: https://github.com/cloudevents/spec/pull/387 Changed: https://github.com/cloudevents/spec/pull/491 Killed: https://github.com/cloudevents/spec/pull/492 Summary: https://github.com/cloudevents/spec/issues/501#issuecomment-532560460

A compressed payload is just compressed payload, e.g. using https://www.rfc-editor.org/rfc/rfc6713.txt