tenant by time

[ghost]

Hello @mwringe Could you please provide me info on what tenant by time table in cassandra represent? Can this be used somehow to get the uptime of a project and then be used to compute % cpu usage at namespace level?

Regards, Shilpa

[mwringe]

@jsanda can you answer the question about what this table is suppose to represent?

[jsanda]

The table is obsolete. It was intended to facilitate basic tenant queries, but a simpler solution was implemented. The table will be removed in a subsequent release.

[ghost]

Is there a way to get project uptime just like how one can get project level cpu usage and memory usage?

[mwringe]

@shilpapadgaonkar How are you calculating project level cpu and memory usage?

[mwringe]

@shilpapadgaonkar Each project does have a 'creationTimestamp' that tells you when the project was created [oc get -o json project $PROJECTNAME]. Not sure if that helps you or not.

[ghost]

To get memory usage from time A(start) to time B(end) for project test I would use

curl -H "Authorization: Bearer XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX" -H "Hawkular-tenant: test" -X GET https://hawkular-metrics.example.com/hawkular/metrics/gauges/data?tags=descriptor_name:memory/usage&buckets=1&start='+ str(start) + '&end=' + str(end)

[mwringe]

Note that without the 'stacked=true' query parameter in there, you are only getting the average usage and not the total usage.

Is there any reason why you couldn't do the same with the uptime metric? This should give the uptime for all the containers running during this time period.

[ghost]

I actually need average..

True.. sounds like a good try.. let me see if the cpu % with this test look ok..

[ghost]

I am trying to get some stacked usage and I get nan records..

First example is without using stacked which gives good result and the next one just adds parameter stacked=true and this gives me nan

@mwringe : Could you please take a look if the query looks ok?

curl -H "Authorization: Bearer xxxxxxxxxxxxxxxxxxxxxx" -H "Hawkular-tenant: atest" -X GET https://172.30.28.145/hawkular/metrics/gauges/data?"tags=descriptor_name:memory/usage&buckets=1&start=1454056894799&end=1454058701234" --insecure | python -m json.tool [ { "avg": 458849065.7622114, "empty": false, "end": 1454058701234, "max": 636641280.0, "median": 476952378.46956563, "min": 105230336.0, "percentile95th": 636641261.8292253, "samples": 1085, "start": 1454056894799 } ]

curl -H "Authorization: Bearer xxxxxxxxxxxxxxxxxxxxxx" -H "Hawkular-tenant: atest" -X GET https://172.30.28.145/hawkular/metrics/gauges/data?"tags=descriptor_name:memory/usage&buckets=1&start=1454056894799&end=1454058701234&stacked=true" --insecure | python -m json.tool [ { "avg": "NaN", "empty": true, "end": 1454058701234, "max": "NaN", "median": "NaN", "min": "NaN", "percentile95th": "NaN", "samples": 23, "start": 1454056894799 } ]

[mwringe]

@stefannegrea any idea why stacked=true would return empty values here?

[stefannegrea]

The non-stacked code collects all the data points from all the matching metrics as originating from a single metric. Thus the bucket has 1085 samples, all looking from a single metric, hence the computed values.

The stacked version first buckets individual metrics. From the number of samples, I can infer there are 23 metrics that match the query. If one the metrics did not have data for the time request then resulting individual bucket for that metric will have a bunch of NaN. When the rest of the buckets are added up, they are still NaN (NaN + any number = NaN).

Note: 1085 is not divisible by 23, that means there are matching metrics without data in the selected interval.

[ghost]

@stefannegrea : If (NaN + any number = NaN), then the stacked search can only work when none of the individual buckets have a NaN. Is this intentional?

[mwringe]

@shilpapadgaonkar is this running the OSE images or the origin-metric ones? We will need to know what versions need to be updated and patched.

The current behaviour with the stacked options is that if any metric returns an NaN value, then the resulting stacked metric will also return an NaN value.

The logic being that if we know the value of A but not the value of B, then the sum of A plus B would also unknown.

In your case, it looks like your query is returning some metrics which didn't collect any data in the time specified. This would be the expected behaviour when a particular container is not running, but it could also potentially be the case that, for whatever reason, metrics were not gathered during this time. Which is why its listed as empty and not as zero.

In some situations it may make sense to consider that if no metrics were gathers (eg the container is not running) then to consider it as zero instead of being empty. This would fix the NaN issue you are seeing when using stacked. We could add a flag to the query parameter to enable this behaviour

[mwringe]

Housekeeping to close older issues. If you think this issue is not resolved, please reopen it.