ribbybibby
commented
4 years ago I haven't pushed a new version since s3_last_modified_object_size_bytes was added. I need to do that.
As for your feature request, templating functionality sounds like an interesting addition as far as it could allow you to capture objects more specifically than just using a prefix. I'll have to experiment with the best way to achieve this. It wouldn't make API calls any less expensive though as the filtering would need to be applied after fetching all the objects. We're limited in that respect by the options supported by the s3 list operation.
A separate interval, divorced from the scrape_interval, for exporters is generally discouraged. Maybe it's justified here because large queries can become expensive and long-running. However, my first instinct would be to leave it as it is and suggest users lengthen the scrape interval and optimise the prefixes they're using.
deployed image: ribbybibby/s3-exporter:latest on kubernetes cluster and s3_last_modified_object_size_bytes metric is missing.
I dont know if you are still maintaing this project, but here is feature proposal: For dynamic filemane check eg. (backup_YYMMDD.csv) prometheus scraping configuration need to be recreated and loaded in order to get up to date metrics/alerts.
It would be interesting add feature to load configuration file with templating functionality? eg.
BackupDate: bucket: BucketA prefix: /backup_{{YYMMDD}}.csv interval:
BackupRandom: bucket: BucketB prefix: /backup_*.csv interval:
which could load multiple bucket checks with logic to limit api calls to S3 with setting the interval (prometheus will scrape with its own interval, but in config file you can set to check for a file in lower intervals eg 1h - not sure how long prometheus holds metrics info - in my case its 6h).
Btw, great work.. and useful.
Regards, Dejan