search

crewAIInc / crewAI

Framework for orchestrating role-playing, autonomous AI agents. By fostering collaborative intelligence, CrewAI empowers agents to work together seamlessly, tackling complex tasks.
https://crewai.com
MIT License
19.25k stars 2.67k forks source link

Ollama Integration doesn't allow setting num_ctx in ollama #638

Closed jcoombes closed 3 months ago

jcoombes commented 4 months ago

Ollama Integration doesn't allow setting num_ctx in ollama

^ The default value of num_ctx in the API is 2k, and I would like to run llama3:70b with num_ctx 8192.

https://docs.crewai.com/how-to/LLM-Connections/

jcoombes commented 3 months ago

closing this - I fixed this by using a MODELFILE where I specified the num_ctx I wanted. This is an ollama feature, not a crewai feature.