search

simulatrex / simulatrex-engine

Enable decision-making based on simulations
https://simulatrex.com
Apache License 2.0
217 stars 20 forks source link

Modify for ollama use instead of just OpenAI #30

Open mpadge opened 1 month ago

mpadge commented 1 month ago

Great piece of software @d42me! It'd be pretty easy to code the two main LLM entry points to allow a range of interfaces instead of just OpenAI. In particular, using ollama would open the whole system up to effectively unrestricted model choice. The completions of ollama are perfectly compatible with openai, and from what I can see of your models.py, the code should need only minimal modification. The embeddings in your vectordb.py would also need minimal modification to use ollama embeddings.

The biggest advantage would be the ability to use unlimited numbers of simulations, as everything would be local and free of additional API costs. Thoughts?

d42me commented 1 month ago

Thanks for suggesting this @mpadge! Will work on this as soon as I find time.