search

opensearch-project / ml-commons

ml-commons provides a set of common machine learning algorithms, e.g. k-means, or linear regression, to help developers build ML related features within OpenSearch.
Apache License 2.0
83 stars 118 forks source link

[FEATURE] Stats REST API to get connector invocation metrics #2476

Open chishui opened 1 month ago

chishui commented 1 month ago

Is your feature request related to a problem? Today, the ML stats API only has total connector count metric, it doesn't report detailed connector invocation metrics. And the algorithm stats can't represent connector usage as for a single algorithm e.g. _predict, we could split docs into chunks and invoke connector multiple times and send multiple requests to remote server.

This connector metrics can help users to have a view on the count and success rate of the connector invocation. It also provides a way for us to have integration tests or end to end test to verify that we invoke the connector correctly.

What solution would you like? We can support new REST APIs:

  1. /_plugins/_ml/connectors/stats
  2. /_plugins/_ml/connectors/{connectorId}/stats

Whenever invoke a connector, record their count and failure count.

What alternatives have you considered? N/A

Do you have any additional context? N/A

dhrubo-os commented 1 month ago

Could you please check profile api if that satisfies your requirement. We have model level prediction count.

chishui commented 4 weeks ago

the count field in profile API is the same with ml_action_request_count of _ml/stats, both measure the model level not the connector level

dblock commented 1 week ago

Catch All Triage - 1 2 3 4 5 6