Metrics on bytes consumed/published

[hkolbeck]

For the time being it seems like it's difficult or impossible to break apart Pub/Sub costs by stream or subscription and so it'd be useful to measure at least roughly client-side. While we can measure bytes written into the client, I don't see a way to measure bytes actually sent post-compression. Our streams vary greatly in compressibility, so this could lead to misleading results. It's also unclear how request overhead is factored in.

Ideally we'd like to measure the following:

• Total raw bytes sent/consumed
• Bytes sent/consumed, rounding batches up to the Pub/Sub minimum request size.
• Count of requests under/over minimum request size.

I can certainly understand reluctance to tie the project to any external metrics library, but perhaps exposing raw data via callbacks or metadata on the returned object would be possible? I'd be happy put together a PR for consideration if this seems like a possibility.

Thanks much, this lib has been a great tool.

[kir-titievsky]

@ckolbeck You might be able to get some of this by looking at breakdowns of "Byte Cost" topic and subscription metrics in Stackdriver. Same for requests. To tie this directly to Pub/Sub bills, I expect a labels feature that will allow you to tag resources with a "label" and export invoices broken down by that label. Not quite per resource, but gets you the granularity you want ultimately.

[hkolbeck]

Oye. I'm not sure how I missed that in Stackdriver. Sorry to waste your time, but thanks for being proxy GCP support.