search

sipcapture / heplify-server

HEP Capture Server for HOMER
https://sipcapture.org
GNU Affero General Public License v3.0
184 stars 85 forks source link

PromTargetIP (Specific + Overall Targets) #407

Closed stephanomarques closed 4 years ago

stephanomarques commented 4 years ago

Hi! When inserting PromTargetIP and PromTargetName, for example:

PromTargetIP = "1.1.1.1, 2.2.2.2" PromTargetName = "Asterisk1, Asterisk2"

My understanding is that heplify-server checks src/dst IP of packet and if matching PromTargetIP it labels with the PromTargetName (target_name="..."). The thing is, I don't want to scrape specifically for every target, so imagine I have 3.3.3.3 and 4.4.4.4 traversing heplify-server but I'm not interested in seeing specific metrics for them but rather overall metrics of both. Is there a way or special char that represents all other IP's so I can get overall and specific metrics, for example:

PromTargetIP = "...,1.1.1.1, 2.2.2.2" PromTargetName = "...,Asterisk1, Asterisk2"

Where ... represents every other IP and is assigned the same target_name so that metrics are being scraped from them and I can still have an overall view of them and specific view for 1.1.1.1 2.2.2.2.

negbie commented 4 years ago

You should be able to do this with prometheus query language. Take a look at sum by or without.

stephanomarques commented 4 years ago

I was aware of that method in PromQL but the problem is there are no metrics apart from the ones scraped from PromTargetIP

PromIP 1 target1 PromIP 2 target2 No target specifically targetnull

When verifying on Prometheus Expression Browser, only metrics from target_name="xxx" (specified in PromTargetName) are shown, although my query is for all "heplify_method_response" entries.

negbie commented 4 years ago

Ah ok now I understand.

negbie commented 4 years ago

Thinking about which label I should use for those metrics. I think it's better to not leave it empty.

negbie commented 4 years ago

Mby unknown?

stephanomarques commented 4 years ago

Unknown sounds good or Other, Any

negbie commented 4 years ago

Ok merged https://github.com/sipcapture/heplify-server/pull/409

negbie commented 4 years ago

Currently unknown metrics are all which does not match targetip do you think we should just make a metric for any or keep it this way?

negbie commented 4 years ago

Basicly every message will increase the any label and additionally target_name if targetip machtes.

stephanomarques commented 4 years ago

Sorry was taking a look at the code and building, looks great to me. Looking at people who will install and use this in the future, I think the approach where there's a metric for Any would be ideal.

stephanomarques commented 4 years ago

Although the benefit of the current manner is it's efficient to select All, Any other than specific or specific metrics.

negbie commented 4 years ago

Ok will think about it. The "problem" with the any metric is performance impact. Currently unknown label will only be set if there is no match src or dst ip. This should be rare cases. But by using any it will hit everytime.

negbie commented 4 years ago

Normally you configure all your edge devices like lb's or gateways as promtarget. So src or dst ip will hit almost everytime.

negbie commented 4 years ago

Mby I should add a new config option like PromMetricFilter. With that stuff like the above could be controlled.

stephanomarques commented 4 years ago

True.. from a performance perspective the "unknown" seems most adequate and for the time being it works. Yes, my issue started from particular cases although almost most are being tagged with PromName.

PromMetricFilter does sound interesting and allows for more flexible configurations. The community can brainstorm some ideas, i'll surely be interested when time is more kind to me.

negbie commented 4 years ago

Sure, feel free to open a new issue if you have some ideas around this!