search

IBM / apiconnect-trawler

API Connect metrics exporter
MIT License
5 stars 10 forks source link

analytics - apicalls in 30 seconds window #56

Closed MLit94 closed 1 year ago

MLit94 commented 1 year ago

At the moment there is a metric in the analytics module to see the apicalls with their http status code only for the last hour. This is not precise enough to use it e.g. for alerts. We need these metrics with a last 30sec window.

rickymoorhouse commented 1 year ago

I think the API we're using for this in v10.0.5 onwards https://github.com/IBM/apiconnect-trawler/blob/b5bdfaf019975e140d050a7c2ac28bf9e30fc31c/analytics_net.py#L212 will support this through specifying start and end times rather than using the timeframe short cut we use today which only supports a minimum interval of last15mins - https://apic-api.apiconnect.ibmcloud.com/v10/#/IBMAPIConnectAnalyticsAPI_200/operation/%2F{analytics-service}%2Fcloud%2Fdashboards%2Fstatus/get

rickymoorhouse commented 1 year ago

I'm exploring adding this as an option in the analytics config of time_interval with a default of 3600 matching the current last hour. If set to 30 this will generate metrics with names like analytics_apicalls_last30s_total, however currently with the way the main trawler loop runs, there is no guarantee this will be every 30 seconds worth of data, so also thinking about an option to query analytics for calls made since the last time the loop ran for when the value is set to 0 - currently I have this metrics like analytics_apicalls_thisloop_total but feel like there might be a better name for that.

To illustrate this potential issue with the looping - if the green bars represent the 'last n seconds' you can see that if n is less than the time taken for the loop to run there is a gap. However if like currently the period the calls are in is bigger than the loop, API Calls are counted for in multiple cycles.

image

The intention of these metrics in trawler wasn't originally to ensure all calls were covered, rather to give an idea of trends, however with smaller intervals it could be more of a concern. The since last run will solve this, but won't always be a regular time period worth of data.

MLit94 commented 1 year ago

Hi @rickymoorhouse , when do you plan to release a new version, so we can use the new intervals?

Greetings Markus

rickymoorhouse commented 1 year ago

Hi @MLit94 I have a couple of other pieces to get sorted and then hope to publish them as a release this week.

rickymoorhouse commented 1 year ago

v1.8.0 has now been published including this item and the missing call to include the gateway_peering metrics

https://github.com/IBM/apiconnect-trawler/releases/tag/v1.8.0