Open pfrybar opened 10 months ago
There are two ways you can get a better view of the number of documents in your setup:
Please let me know if this helps you accomplish what you want.
Best, -Yngve
Thanks for the quick response. I wasn't able to find a "groupId" dimension on any of the metrics, whether using the aggregated v2
metrics, node-level v1
metrics, or the prometheus endpoint.
I could try to do some aggregations on "distributor.vds.distributor.docsstored", but since we are using multiple content groups and I can't find a dimension to group by I'm not sure how to do it in a generic way.
For example, here is /state/v1/metrics
on a content node:
{
"name": "content.proton.documentdb.documents.total",
"description": "The total number of documents in this documents db (ready + not-ready)",
"values": {
"average": 9743208.0,
"sum": 116918496.0,
"count": 12,
"rate": 0.2,
"min": 9743208,
"max": 9743208,
"last": 9743208
},
"dimensions": {
"documenttype": "mydocument"
}
},
You should not use the groupId
dimension for the distributor metric, but sum over them. If you have multiple content clusters then you will need to aggregate per cluster using the clusterid
dimension.
Unfortunately it seems metrics are only decorated with those dimensions in the Vespa Cloud currently.
I will clear assignee to have this discussed during our upcoming ticket scrub.
Is your feature request related to a problem? Please describe. When creating monitoring dashboards, it's difficult to know how many documents exist when using a grouped distribution, from just metric data alone. We can use the current metrics to find the total documents across the entire cluster, but there is no way to find the "correct" (average) number of documents within a content group.
Describe the solution you'd like Expose a metric for the number of content groups currently in use.
Describe alternatives you've considered We have graphs of total document count which show a rough picture, but they get noisy when the number of content groups change. See the screenshot below as an example: