search

vespa-engine / vespa

AI + Data, online. https://vespa.ai
https://vespa.ai
Apache License 2.0
5.58k stars 586 forks source link

when conditional update is not met it's visible with REQUEST_ERROR metric #23075

Closed shelliglick closed 2 years ago

shelliglick commented 2 years ago

When conditional update is not met, a REQUEST_ERROR metric increases. For example:

feed_operations_rate{api="documentV1", deploymentType="stable", operation="UPDATE", pod="vespa-stateless-container-0", service="vespa-stateless-container-monitoring", status="REQUEST_ERROR", tenant_id="default-tenant", vespaVersion="7.524.43", vespa_service="vespa_container"}

This is confusing, as it sounds like some error. Should probably be changed to a different and more related label.

Also, not directly related to this ticket - it would be VERY useful if the feeding request would get back in the response what was the actual result of the update. For example if no update took place due to a not met condition (or if partial update had no impact due to a missing document).

kkraune commented 2 years ago

I tested this myself using https://docs.vespa.ai/en/operations/monitoring.html and observed the same in the prometheus graphs. I think we should add the expected behaviour to https://docs.vespa.ai/en/reference/metrics.html

CC @gjoranv the metrics guru

bratseth commented 2 years ago

We should not increase REQUEST_ERROR on condition-not-met.

olaaun commented 2 years ago

This issue has been fixed in PR #23115, which will be included in Vespa version >= 8.2.17