search

thumbor-community / prometheus

MIT License
8 stars 8 forks source link

Add multiprocess support #16

Closed VladVolchkov closed 2 weeks ago

VladVolchkov commented 9 months ago

relates to #15

deepminimal commented 7 months ago

@savar @mbarouski Hello! Can you check it please?

savar commented 7 months ago

@savar @mbarouski Hello! Cat you check it please?

Hey, I am not experienced in this multiprocess part of the prometheus client. I am trying to read the doc but I am not sure that this Collector stuff can be done only once in the initializer as the example in https://prometheus.github.io/client_python/multiprocess/ is doing this every time the scrape is happening.

Also as far as I can tell the multiprocess would require a properly set PROMETHEUS_MULTIPROC_DIR environment variable to work. Maybe we need to test for this envrionment variable to be set and only if, then we start the multiprocess part, otherwise we stick to the existing setup?

Do you have a working example deployment where this is being used, so that we could have a better look onto it?

savar commented 7 months ago

It looks like it could work but we will lose the python_* and process_* as well as the *_created metrics. Also I would like to not use this when there is only a single process being used, specifically because we do not execute the CollectorRegistry+MultiProcessCollector on each scrape request but only once initially. The documentation states it has a risk, but I cannot say right now if this applies to us here.

I just cannot figure out how we could detect the multiprocess situation. Thumbor is not providing this information in the config it passes into the initializer and relying on the environment variable would only mean that people do not realize that they didn't configure it correctly. :thinking:

sboisson commented 4 months ago

Maybe only use CollectorRegistry and MultiProcessCollector when PROMETHEUS_MULTIPROC_DIR env var is set ?