opensearch-project / ml-commons

ml-commons provides a set of common machine learning algorithms, e.g. k-means, or linear regression, to help developers build ML related features within OpenSearch.
Apache License 2.0
99 stars 136 forks source link

Check before delete #3209

Open xinyual opened 1 week ago

xinyual commented 1 week ago

Description

This pr is to check all downstream service before deleting ml model. Two downstream tasks are checking here:

  1. Agent
  2. Pipelines

For agent, we enforce this tool factory to implement a method to return the key field name of each tool. Then we create a should DSL query like

{
  "query": {
  "should": [
  "terms": {
            "tool.parameters.model_id": ["delete_model_id"]
        },
    "terms": {
            "tool.parameters.embedding_model_id": ["delete_model_id"]
        }
  },
  "terms": {
            "tool.parameters.inference_model_id": ["delete_model_id"]
        }
  }]
}

For pipelines, we fetch all ingestion pipelines and search pipelines and check for each pipeline whether they contain the candidate model id.

Related Issues

3191

3087

3088

Check List

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. For more information on following Developer Certificate of Origin and signing off your commits, please check here.

yuye-aws commented 4 days ago

@xinyual You can raise another PR in the skills repo. It's better we can review both PRs togeter.

xinyual commented 4 days ago

@xinyual You can raise another PR in the skills repo. It's better we can review both PRs togeter.

Sure.

xinyual commented 4 days ago

Please update DeleteConnectorTransportAction with connector_id in ConnectorTool.

It's not related to this PR. I will check it and open another PR if it's need.