search

empirical-run / empirical

Test and evaluate LLMs and model configurations, across all the scenarios that matter for your application
https://docs.empirical.run
MIT License
141 stars 10 forks source link

Google Vertex AI Integration Idea #236

Open arjunattam opened 1 month ago

arjunattam commented 1 month ago

Discussed in https://github.com/empirical-run/empirical/discussions/235

Originally posted by **TalkTomeG00se** May 29, 2024 Hello! First I want to say thank you for this awesome tool, really enjoy it's flexibility and ease of use. This is part idea and question. I think integration somehow with the Vertex AI platform would be great. I do see that Empirical supports Gemini models via a Google API key. Those API keys are generated through access to Google AI Studio, which in turn is accessed through Google Workspaces. In my instance however, my company doesn't leverage Workspaces, but instead has access to Vertex AI within Google Cloud, and access to that API is not done through API keys, but through IAM policies/application default credentials. The nice thing about using the Gemini API for Vertex AI, is that we have access to dozens of models including all the Google of course, Anthropic, Mistral etc. Not sure if that can be done, but wanted to throw that idea out there. Thanks again for a amazing tool, keep up the amazing work!

cc @talktomeg00se

arjunattam commented 1 month ago

Thanks for the request. How do you authenticate your requests to Vertex AI endpoints? Based on this guide, I see that there are few different approaches. It'll help to know which one to prioritize.

TalkTomeG00se commented 4 weeks ago

We leverage Application Default Credentials to access Vertex AI right now. Though if one implementation is easier to implement within Empirical than another, we can do that too, we just use ADC right now.