Crew AI will not run with purely opensource solutions

[JamesStallings]

Crew AI, by default, uses OpenAI models that require a paid up account and API keys. There may be some free tier, but I am profoundly disinterested in being rate limited or exhausting some meager free tokens resource, much less have some or all of my prompts delivered into the hands of OpenAI.

This is a known issue, of which the author of Crew AI has been repeatedly informed, and he lacks sufficient imagination to understand that the use-case of a purely localized, purely open source solution is a viable use case. Please do not promote Crew AI as an Open Source or cost-free solution to agent marshalling.

[premthomas]

Hi James. Thank you for your observation.

I understand and feel the frustration behind using OpenAI models. Besides not having the resources to test LLMs that have some cost associated with them, I had to limit my use of cloud platforms like AWS and GCP, and resort to creating and testing my research on my PC locally.

Could you please clarify what you meant by "purely open source solutions"? This blog was written to show how I used models from Ollama with CrewAI. I am in no way associated with any of these companies except to learn the different ways to get tasks done.

Please let me know if you know of other agent tools. I would love to read and implement some of my other ideas using them.

[JamesStallings]

By that I mean that if you attempt to use crew AI with e.g., llama3 which has been installed locally with ollama, Crew AI still demands an API key for Open AI. Apparently, this has to do with Crew AI's insistence on using Open AI's embeddings.

I am no authority on these technologies, if you have a method by which I can substitute embeddings from a different source, for instancec embeddings I have generated using ollama and llama3, I would be quite delighted to hear of them.

Thank you for acknowledging my concerns :)

Cheers James

[premthomas]

James, I replaced the "openhermes" model with "llama3" and I did not get any errors or prompts asking me for an API_KEY. I will re-review my code to see if there is something I am missing but can I ask that you check the code that I have attached to this post?

I have not explored using a custom/retrained LLM and I hope to start that soon. I did find something on the CrewAI docs that also suggests that the standard Ollama models can be used without the API_KEY. Here is the link. CrewAI does not tell us how the model is "custom".

If you have come across any other "agent" tools as an alternative to "CrewAI", please let me know so that I can explore them. I am hoping that it would be something that I will not need an API_KEY for.

Thank you again for your observations!

[JamesStallings]

Greetings Prem 🤓

I too had read that an OpenAI API key was not a requirement for using Crew AI. However when I tried to use it as suggested for running local models, and without such a key, it threw an internal exception for lack of the key.

Concerning running the models locally, you really just need one each of sufficient hardware and ollama open source software. Ollama has both python bindings and a locally served API, both of which abstract the various supported LLM under a common software interface. The hardware requirements, much like the ability of the models, become more agreeable on a near weekly basis; I’m running debian 12 on a cybergeek ‘nuc’, with an 8 core hyper-threading AMD Ryzen 7 with 64gb ram and 2 tb nvme, and I have all of 500$ in it, and while it’s a bit more than the minimum requirements, it gives me room and threads to run a few or three instances of the same model, the system remains responsive with plenty of room and cycles to run the os and assorted dev tooling. I wouldn’t call it blazing fast; but it isn’t exactly slow, either. Bonus: if you really make it grind the gears, it might use most of 20 watts 🤪

For the next few days, I am obligated to focus on work that pays directly, but beyond that I will have a nice block of time to focus on my AI project(s), which for now are pure science, technical research, and financially speculative all at once 😝

I will bring the Crew AI concern topmost, as right now in this context I am attempting (in a fairly feeble way, I must say), to roll my own solution, vs the Crew AI ready-made.

I do so hate reinventing the wheel, however much it might teach me about axles 🥸.

You can count on me to update vs. my findings.

Thank you once again for seeing my concerns 😌

Kindest regards, James

ॐ♲ ♥ ☸☮ ☯☰☶☯☮ ☸ ♥ ♲ॐ KI5SMN

On Mon, Jun 24, 2024 at 9:47 AM Prem Eapen @.***> wrote:

James, I replaced the "openhermes" model with "llama3" and I did not get any errors or prompts asking me for an API_KEY. I will re-review my code to see if there is something I am missing but can I ask that you check the code that I have attached to this post?

I have not explored using a custom/retrained LLM and I hope to start that soon. I did find something on the CrewAI docs that also suggests that the standard Ollama models can be used without the API_KEY. Here is the link https://docs.crewai.com/how-to/LLM-Connections/#ollama-integration-ex-for-using-llama-2-locally. CrewAI does not tell us how the model is "custom".

If you have come across any other "agent" tools as an alternative to "CrewAI", please let me know so that I can explore them. I am hoping that it would be something that I will not need an API_KEY for.

Thank you again for your observations!

— Reply to this email directly, view it on GitHub https://github.com/premthomas/Ollama-and-Agents/issues/6#issuecomment-2186759647, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAA3PVRP65YNVVB7B3NV7G3ZJAWQXAVCNFSM6AAAAABJP7YJJKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOBWG42TSNRUG4 . You are receiving this because you authored the thread.Message ID: @.***>

[JamesStallings]

Hey there Prem,

I just wanted to drop you a line and let you know that on a more recent version of Crew AI, I have not had the issues with respect to OpenAI API Keys.

On Wed, Jun 26, 2024 at 6:37 AM Prem Eapen @.***> wrote:

Closed #6 https://github.com/premthomas/Ollama-and-Agents/issues/6 as completed.

— Reply to this email directly, view it on GitHub https://github.com/premthomas/Ollama-and-Agents/issues/6#event-13296989311, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAA3PVXG4WKPIFJME26QRQDZJKRWXAVCNFSM6AAAAABJP7YJJKVHI2DSMVQWIX3LMV45UABCJFZXG5LFIV3GK3TUJZXXI2LGNFRWC5DJN5XDWMJTGI4TMOJYHEZTCMI . You are receiving this because you authored the thread.Message ID: @.***>

-- ॐ♲ ♥ ☸☮ ☯☰☶☯☮ ☸ ♥ ♲ॐ KI5SMN