opensearch-project / flow-framework

OpenSearch plugin that enables builders to innovate AI apps on OpenSearch
Apache License 2.0
32 stars 31 forks source link

[FEATURE] Support internal connector in register model #212

Open amitgalitz opened 9 months ago

amitgalitz commented 9 months ago

Is your feature request related to a problem?

Right now we have a step called RegisterRemoteModel in our framework that correlates to the register_model API In ml-commons when a remote model is given. Currently we only support creating a remote model with a given connector ID. For example:

POST /_plugins/_ml/models/_register
{
    "name": "openAI-gpt-3.5-turbo",
    "function_name": "remote",
    "model_group_id": "1jriBYsBq7EKuKzZX131",
    "description": "test model",
    "connector_id": "a1eMb4kBJ1eYAeTMAljY"
}

However, ml-commons also supports creating an internal connector inside the model itself through this API. We currently don't support this, example payload:

POST /_plugins/_ml/models/_register
{
    "name": "openAI-GPT-3.5: internal connector",
    "function_name": "remote",
    "model_group_id": "lEFGL4kB4ubqQRzegPo2",
    "description": "test model",
    "connector": {
        "name": "OpenAI Connector",
        "description": "The connector to public OpenAI model service for GPT 3.5",
        "version": 1,
        "protocol": "http",
        "parameters": {
            "endpoint": "api.openai.com",
            "max_tokens": 7,
            "temperature": 0,
            "model": "text-davinci-003"
        },
        "credential": {
            "openAI_key": "..."
        },
        "actions": [
            {
                "action_type": "predict",
                "method": "POST",
                "url": "https://${parameters.endpoint}/v1/completions",
                "headers": {
                    "Authorization": "Bearer ${credential.openAI_key}"
                },
                "request_body": "{ \"model\": \"${parameters.model}\", \"prompt\": \"${parameters.prompt}\", \"max_tokens\": ${parameters.max_tokens}, \"temperature\": ${parameters.temperature} }"
            }
        ]
    }
}

What solution would you like?

There can be two solutions for this, either we create a new step name called register_model_with_internal_connector/register_internal_connector/register_internal_model. Or we change the register_remote_model step to allow for either a connector ID or a connector object by changing the parsing method here: https://github.com/opensearch-project/flow-framework/blob/main/src/main/java/org/opensearch/flowframework/workflow/RegisterRemoteModelStep.java#L109

To satisfy what a connector object is, we can potentially refactor some of the code in the CreateConnectorStep so we don't have duplicates for the connector parsing.

dbwiddis commented 8 months ago

This relates to #339 implemented in #340 where I combined the register + deploy steps. This would be really nice to include.