Use protobuf encoding for core K8s APIs in Prometheus Engine

[tosi3k]

For core K8s API objects like Pods, Nodes, etc., we can use protobuf encoding which, compared to the default JSON encoding, reduces CPU consumption related to (de)serialization, reduces overall latency of the API calls, reduces memory footprint and the work performed by the GC and results in quicker propagation of objects to event handlers of shared informers.

Core system components of K8s default their serialization method to protobuf for 8 years already: https://github.com/kubernetes/kubernetes/pull/25738.

Some benchmarks comparing JSON vs. protobuf showcasing how the latter data format (de)serializes faster and uses less memory:

• https://medium.com/@akresling/go-benchmark-json-v-protobuf-4ec3c62ec8d4
• https://www.codingexplorations.com/blog/performance-comparison-protobuf-marshaling-vs-json-marshaling-in-go
• https://medium.com/@david_turner/protocol-buffers-just-how-fast-are-they-9ec19ea580db

[tosi3k]

Is there a case where potential non-GKE k8s would ONLY support JSON and this will fail or client is able to fallback?

It would only fail in K8s older than 1.3 in which protobuf support was added. It's embedded in the server and can't be turned off.

[bwplotka]

Thanks!