Closed Lusitaniae closed 1 month ago
@Lusitaniae I have one query with this. I understand you want to expose it in compressed format but can you name few metrics collectors that scrape in compressed format and decompresses it? I have worked with Telegraf and chronocollector and both don't support scraping metrics that are in compressed format
I guess if this is not the standard then not sure if its a genuine feature request
I see For now it seems only gzip format is supported Maybe I can pick this up in the following week
Much appreciated :pray:
@Lusitaniae This would require to contribute to https://github.com/prometheus/client_ruby/blob/main/lib/prometheus/client/formats/text.rb the official prometheus client for ruby. The hackaround for this is we break the abstractions in the Fluentd prometheus plugin to fetch the metrics info and compress and host directly. Not sure which is a better call because maybe first is overkill but I feel that's the write method
Doesn't this solve it?
It's highly recommended to enable gzip compression for the metrics endpoint, for example by including the Rack::Deflater middleware.
https://github.com/prometheus/client_ruby?tab=readme-ov-file#rack-middleware
Oh yah i missed this infact. Thanks for pointing it. I will try to check this in my free time.
I was checking this @Lusitaniae. So for the Rack middleware to be used we need to have the app(http server) exposed and get overriden with the middleware. But i guess the code abstraction in fluentd has made use of a plugin helper so seems like its not as direct as we see. I am actively checking how to make use of it but incase if you also find something interesting please let me know
For now i am not using rack but trying to compress the data directly and put it in the response body
@kenhys the PR is done for this and reviewed by @ashie. I guess u can change the label to the appropriate one https://github.com/fluent/fluent-plugin-prometheus/pull/223
Maybe can close this one, since it was implemented on the plugin itself, and I also filed an issue there
https://github.com/fluent/fluent-plugin-prometheus/issues/219
Is your feature request related to a problem? Please describe.
I'm using Prometheus plugin to export metrics based on logs ingested by fluentd.
In some servers these metrics can amount to millions of lines, which reach payload sizes of 50/60Mb and either reach the scraping limit (set by vmagent), or they failed to be transitted within the timeout period (10s)
Compressing data would be a clear winner to solve this problem
Describe the solution you'd like
Allow prometheus plugin to send compressed responses
Describe alternatives you've considered
Bumping payload limits / timeouts
Additional context
Cross posted from https://github.com/fluent/fluent-plugin-prometheus/issues/219 in case this is the right place to implement such feature