search

nzin / prometheus-cachethq

Prometheus CachetHQ bridge
MIT License
6 stars 9 forks source link

Create Cachet component if not exist #4

Open lukasmrtvy opened 5 years ago

lukasmrtvy commented 5 years ago

I have feature request :)

I am wondering if something like -e FORCE_CREATE_COMPONENT_IN_CONSUL=true can be used for explicit component creation if not exists in Cachet.

When You have dynamic environment like K8s, its very hard to keep components 1:1 to Your services in K8s. I know that proper way should be to register/unregister services in Consul,Etcd,etc and then dynamicaly create/delete components in Cachet, but this can be a starter point for this type of problem.

Thanks

nzin commented 5 years ago

Hi Lukas, yeah, creating component on the fly, was something I wanted to implement but never took the time for. Let's see how we can do that (after the CA certificate improvement).

By the way, and just to be clear, I know how to create new components, but I dont think it is easy to know when to remove one... Prometheus alert manager cannot report when a component is removed: it doesnt know that.

lukasmrtvy commented 5 years ago

Something like Docker / Swarm / K8s discovery would be a good solution. (Also Consul and ETCd) You can scrape Pods/Containers from containerd socket with specific label (--label cachet-discovery=true) a dynamically create/update/destroy components with prometheus-cachethq in Cachet.

nzin commented 5 years ago

I was more expecting to create dynamically components when they are not present. But to delete them, I wouldn't want to do it automatically:

lukasmrtvy commented 5 years ago

I am aware of this.. Desired versus actual list of services.

Lets start with on the fly creation :)