search

weaviate / Verba

Retrieval Augmented Generation (RAG) chatbot powered by Weaviate
BSD 3-Clause "New" or "Revised" License
6.09k stars 651 forks source link

Integrate Embedding Models Using Vertex AI with Service Account Authentication #189

Open dawnpatrol04 opened 4 months ago

dawnpatrol04 commented 4 months ago

Description

Due to security policies, embedding API keys are not permitted in most companies. We need to use the Vertex AI service account (gemini_credentials.json) for authentication when integrating embedding models. This approach ensures compliance with security standards and leverages the benefits of using service accounts for authentication. This also simplifyes the config process becasue all that is requred for people using GCP ( most big companies currently ) is the service account / gemini_credentials.json.

Is this a bug or a feature?

Steps to Reproduce

  1. Setup Vertex AI Service Account:
    • Ensure gemini_credentials.json is correctly configured and accessible.
    • Verify that the service account has the necessary permissions for accessing embedding models.
  2. Modify Authentication Method:
    • Update the codebase to use gemini_credentials.json for authentication instead of embedding API keys.
    • Implement the necessary changes in the configuration files to support service account authentication.
  3. Testing and Validation:
    • Conduct thorough testing to ensure that the embedding models function correctly with the service account authentication.
    • Validate that all functionalities are working as expected and that there are no security issues.

Additional context

This change is necessary to comply with company security policies, which prohibit the use of GCP API keys. By using a service account (gemini_credentials.json), we ensure that our authentication method is secure and aligns with industry best practices.

dawnpatrol04 commented 4 months ago

Hey if someone can point me in the right direction I can probably get a pull request together.