search

andrewyng / translation-agent

MIT License
4.83k stars 553 forks source link

Use free Llama 3 on Groq instead of OpenAI #16

Open jeffxtang opened 5 months ago

jeffxtang commented 5 months ago

If using OpenAI is too expensive for you to try out examples with long text, you can create a free API key in seconds at https://console.groq.com/keys using your gmail or github then make the changes below:

  1. run: pip install groq

  2. Add a new line in your .env file (renamed from .env.sample, originally with a line starting with "OPEN_API_KEY"): GROQ_API_KEY="your_groq_api_key"

  3. In src/translation_agent/utils.py, make 3 changes:

    • change import openai to from groq import Groq
    • change client = openai.OpenAI(api_key=os.getenv("OPENAI_API_KEY")) to client = Groq(api_key=os.getenv("GROQ_API_KEY"))
    • change model: str = "gpt-4-turbo" to model: str = "llama3-8b-8192" or model: str = "llama3-70b-8192"

Now running python examples/example_script.py will use Llama 3 on Groq for free.

icecreamzhao commented 5 months ago

Add one more line: change model_name="gpt-4" to model_name="llama3"

methanet commented 4 months ago

@jeffxtang Thanks for the suggestions! Adding support for Llama would be a great extension to the codebase. Would you like to submit a PR with the proposed extension incorporating the Groq APIs? Much appreciated.

@icecreamzhao good catch!