GCP Vertex AI API support

[brockWith]

Is your feature request related to a problem? Please describe.

I want to utilize various LLM APIs like GCP or Claude API, but I only see OpenAI right now. If you can add other APIs, I would like to be able to compare outputs across LLM vendors.

Describe the solution you'd like Add GCP Vertex AI API(PaLM2), Claude API etc to model variant

Additional context

[aakrem]

Hi @brockWith, hopefully soon. We have already a PR in the same context https://github.com/Agenta-AI/agenta/pull/790/files

[mmabrouk]

Hello @brockWith, you can already create LLM variants using any of these models. You can do that by writing the code for the LLM application yourself. For instance here is an example using claud (untested):

import agenta as ag
import requests

ag.init()
ag.config.default(prompt=ag.TextParam("what is the capital of {country}"),
                              max_tokens=ag.IntParam(100),
                              temperature=ag.IntParam(100))

@ag.entrypoint
def generate(country:str):
    response = requests.post(
        'https://api.anthropic.com/v1/complete',
        headers={
            'accept': 'application/json',
            'anthropic-version': '2023-06-01',
            'content-type': 'application/json',
            'x-api-key': 'your_anthropic_api_key' 
        },
        json={
            'model': 'claude-2',
            'prompt': ag.config.prompt.format(country=country),
            'max_tokens_to_sample': ag.config.max_tokens,
            'temperature': ag.config.temperature
        }
    )
    return response["completion"]

As @aakrem mentioned, we are working on adding these to the UI templates too.

Please let me know if you need any help deploying these models or if you find any problems there.

[brockWith]

Thanks!!!