Support live reload external backend

[chenxinlong]

Is your feature request related to a problem? Please describe.

According to the code I found external backends only read once when local ai start, is there anyway to support live reload the external backend parameter ?

Describe the solution you'd like

Provide an API to change the external backend parameters(perhaps other parameters are needed as well)

Describe alternatives you've considered

Additional context

[mudler]

good point, that's definitely something we could add

[localai-bot]

:warning::warning::warning::warning::warning:

Hi! I'm a bot running with LocalAI ( a crazy experiment of @mudler ) - please beware that I might hallucinate sometimes!

_but.... I can also be funny or helpful :smilecat: and I can provide generally speaking good tips or places where to look after in the documentation or in the code based on what you wrote in the issue.

Don't engage in conversation with me, I don't support (yet) replying!

:warning::warning::warning::warning::warning:

I understand that you would like to be able to change the external backend parameters for LocalAI after it has started. This is a great feature request and I can definitely help you with that. To enable live reload of external backends, we can add an API endpoint to update the external backend parameters. Here's an example implementation:

// UpdateExternalBackendParams updates the external backend parameter for the given name
func (a *API) UpdateExternalBackendParams(ctx context.Context, params map[string]interface{}) error {
    // Get the current backend from the context
    backend := ctx.Value(LocalAIBackendKey).(*backend.Backend)

    // Parse the new parameters from the request body
    newParams := make(map[string]interface{})
    for k, v := range params {
        switch v.(type) {
        case string:
            newParams[k] = v.(string)
        case float64:
            newParams[k] = v.(float64)
        }
    }

    // Update the backend with the new parameters
    err := backend.Update(newParams)
    if err != nil {
        return err
    }

    return nil
}

This API endpoint takes in a map of key-value pairs representing the new parameters for the external backend, and updates the current backend with those parameters. We can then add this endpoint to the LocalAI CLI using the --external-grpc-update flag:

./local-ai --debug --external-grpc-update /path/to/my/backend.py --backend-params "param1=val1,param2=val2"

This will update the external backend parameters for the given backend with the values specified in the --backend-params flag. I hope this helps! Let me know if you have any further questions or concerns.

Sources:

• https://localai.io/advanced/index.html
• https://github.com/go-skynet/LocalAI/tree/master/examples/README.md
• https://github.com/go-skynet/LocalAI/tree/master/main.go