seyeong-han commented 1 month ago

Motivation

I wanted to participate more in solving the listed issues, but I already spent more than $30 on debugging with the ChatGPT API, lol.

Recently, Mistral announced that they have reduced their API prices by up to 80%. I was curious to find out which model is the most affordable. Therefore, I created a table to compare the prices of different models in order to determine which API would be the cheapest to implement.

Model	Input Price (/M tokens)	Output Price (/M tokens)
Mistral Nemo	$0.15	$0.15
Mistral Small	$0.2	$0.6
Mistral Large	$2	$6
GPT-4o mini	$0.15	$0.6
GPT-4o-2024-08-06	$2.5	$10
Gemini 1.5 Flash	$0.075	$0.3
Gemini 1.5 Pro	$3.5	$10.5

To advocate participation without having cost concerns, I want to implement Mistral LLM models, which are way cheaper than the current GPT-4o API.

I know that using a small model hinders agents' performance, but it would be good for fast debugging and improving features.

kingsotn-twelve commented 1 month ago

you could check out Groq too, which offers free llama-3 models that can be useful for debugging

https://console.groq.com/keys

seyeong-han commented 1 week ago

Updated intention for this issue

I found the local LLM model would be good enough for Jockey developers to test this library. So, I decided to integrate the llama3 model using Ollama for this Jockey project.

Good stuff to cononect Ollama and Langgraph docker networks. genai-stack

james-le-twelve-labs commented 22 hours ago

@seyeong-han Thank you! Do you want to start working on a PR for this issue?

twelvelabs-io / tl-jockey

Support Llama3 with Ollama #81

Motivation

Updated intention for this issue