Still Anthropic API key error

[MedleMedler]

I also get this error message, after exactly following your API instructions:

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Error occurred when executing Griptape Agent Config: Anthropic:

No API key provided. You can set your API key in code using 'voyageai.api_key = ', or set the environment variable VOYAGE_API_KEY=). If your API key is stored in a file, you can point the voyageai module at it with 'voyageai.api_key_path = ', or set the environment variable VOYAGE_API_KEY_PATH=. API keys can be generated in Voyage AI's dashboard (https://dash.voyageai.com).

File "F:!COMFYUI\ComfyUI_windows_portable\ComfyUI\execution.py", line 151, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

I red the previous message about Antropic API key and I now understand that putting the Anthropic API Key in the .env file, does not work anymore.?

Will you make a workaround, or are we stuck to use Voyage AI? If we are stuck to Voyage AI, can you please make a tutorial how to exactly make this work with Voyage AI?

Maybe off topic, but do I understand correctly that you have to pay Anthrophic for generated ai data and now also have to pay Voyage AI for generated ai data that goes through their API embedding model?

Fur sure off topic (but worth mentioning) I already saw that Voyage Ai (and also Open Ai) only accepts credit cards. That's a big disappointment for everybody without credit card, they then also have pay credit card fees. Pay, pay, pay,... anyway.

[shhlife]

Hi! I have a fix for this coming tomorrow that lets you choose a different embedding driver, just doing some final testing and then I’ll release it. :)

[shhlife]

Here's an example with the upcoming release where I'm using Anthropic as the prompt driver, and I don't have anything else defined. My .env file only has the ANTHROPIC_API_KEY in it, so there's no embedding driver defined.

[shhlife]

Okay, I've just pushed the latest update. Please check it out and let me know what you think!

[MedleMedler]

Hi Jason thank you very much. I will have a look at it today and give you feedback. :-)

[MedleMedler]

Ok, I get this message: Error code: 400 - {'type': 'error', 'error': {'type': 'invalid_request_error', 'message': 'Your credit balance is too low to access the Claude API. Please go to Plans & Billing to upgrade or purchase credits.'}}

Seams like it is working fine. As soon as I am up and running, I will let you know.

[shhlife]

Cool - thank you very much!

[MedleMedler]

Your welcome Jason. I would like to become involved with Griptape. This is a diamond in the rough. Griptape could make Confyui a platform that connects everything wit anything. The key to unlock Confyui (and make it "Comfyui Everything") is more tools, tools and tools.

Your file management tool is a great step in the good direction. But what about a file manger tool plus, (e.g. for moving, copying and deleting files). With File Manager Plus, you can make your Griptape Agent a database manager, who can organise, rearrange files, asses files (e.g.: delete all images below a certain aesthetic image quality based on a set of rules (E.g. to automate your back end office for all the generated images)).

The sky is the limit Jason. Griptape could make Confyui a platform where all disciplines merge together, giving everyone unprecedented control in building their own sophisticated agent workflows (merged with all other Confyui nodes) for any business, any art, any specialism, any profession and/or any field.

I assume you have discord, where one can have more in depth conversations about Griptape?

[shhlife]

Thanks for the feedback! I'm really enjoying the response people are having to the integration of the Framework into a system like ComfyUI - it's been amazing. :)

We do have a Discord, you can get to it here from ComfyUI by clicking with the Right Mouse Button and going to the Discord menu:

Or, you can just go here: https://discord.gg/griptape

[MedleMedler]

Hi Jason. I am trying Gripetape out. First of all, I am learning a lot about the LLM's (chatbots) to get a good grasp of what is possible and what not. It is difficult to give good feedback, if you do not know how ai's work through Griptape in depth and where Griptape stands in usability in general.

But for now I can share this:

• the Ollama config module is not always consistent and sometimes gives a blank field in the "prompt model" field. It can suddenly dissapear if you change something (getting an error message) and you can not select the Lama Model in that field anymore. Which means you have to delete this module, select a new one and start all over again.
• I also notice that the ChatGTP 4o ai can give error message that their system is overloaded (to crowded). Nothing you can do about this, but has indirectly influence on the experience of using Griptape. Paying money for using ChatGTP 4o and getting errors for overcrowding is annoying makes it inconsistent.

[MedleMedler]

Another one: When I use ChatGTP 4o (with its configuration model connected to the "create agent module"), ChatGTP runs every generation, although I have set "control after generate" option to fixed. This is un practical because the ai every time the ai generates a new answer, losing the old one.

It would be wonderful if you have long connected forks with several "create agent modules" (every agent connected with his own configuration model) and you can set every agent to "fixed", " or "random" (with each agent his own seed).

[shhlife]

Hi @MedleMedler

Yes, I've been noticing the issue with Ollama as well. I'm finding that llama3 is working more consistently than 3.1 - which is odd. It's also having issues with Tools - I've added a ticket to track that internally.

As for the prompt model field, I just pushed an update this morning that should hopefully solve the model missing from the input - please let me know after your latest update if that fixes it.

With ChatGPT4 getting overloaded - This is very frustrating, I know. I'm hopeful they'll resolve the issues soon! you can always try going to 4o-mini, 4, or 3.5 to see if that takes care of things for a little while?

Also, I set the default control_after_generate to fixed now for all config nodes - this will hopefully solve the re-generation problem!

Please take a look and let me know if these fixes work for you.

cheers, Jason

[yudhisteer]

Hi, where should we put our .env?

[MedleMedler]

Hi Jason,

I have been very busy (and will be for the next period). As soon I am able to test it, I will let you know. :-)

[shhlife]

@yudhisteer you can put the .env file in your base ComfyUI directory. Or, you're welcome to use the Griptape Agent Config: Environment Variables node and save your api keys there. :)

@MedleMedler - I'm closing the ticket for now, but please let me know when you're able to test!

[MedleMedler]

Hi Jason,

I finally had some time to evaluate:

• The nodes work better now.
• The "Griptape Agent Config: Ollama" crashed a few times, but much less.
• I would like more output ports, so the ai can diversify its outputs and will give the agent nodes much more functionality. Now I let ai make 1 total output string (with multiple sub outputs) and then do tricks with text nodes to separate the sub outputs. This is sensitive for errors, because the ai model has to be trained enough to be consistent in the output format. An other option would be the use of multiple agents, but this seams more time consuming and slows the process down.
• Two days ago Meta came with a new open source ai model (hermes3) that blows my socks off. There was a lot of attention for llama 3.1. , because this was important for Meta to let everybody know. Now there is not so much attention for hermes3, but don't let the low attention mislead you. Hermes3 is by far much better then Llama3.1, is very consistent, can act very autonomous and has incredible visual analysing qualities. It's visual analysing qualities are so good, that you have an aesthetic scorer out of the box. E.g. you simply can ask hermes3 to output a score and can use this value to to NOT save any images below a certain value.
• What I also discovered is that if you save the model (e.g. command: Ollama cp hermes3 hermes3_trained), you save a copy of the trained model, without writing any code. It seems like this improves the output quality, but I am not sure yet. Could well be this is not the case; do you know if you can save trained models (by using them in Confyui) this way?

I would like to advise you to take this with you in your next youtube videos. :-)

I will keep contact using your Discord page in future and am curious how Griptape Confyui will develop. :-)

[MedleMedler]

I forgot to mention, that I don't know if hermes3 is intelligent enough to handle complex tasks (and dito GripTape workflows) like chatgtp can and (Llama 3.1 not). It wouldn't surprise me if hermes3 can and there will be no more need for separate GripTape workflows. For sure you will try hermes3 out by now. Please let me know if herems3 can also handle more complex GripTape workflows, like Chat gpt and Claude can.

[shhlife]

Thanks for the notes! I’ll give hermes3 a try! You may be able to use the OpenAiCompatible nodes and ping their server..

https://docs.lambdalabs.com/on-demand-cloud/using-the-lambda-chat-completions-api

If you can replicate the situations where you find ollama crashing, I’d love to try and solve it.

as for multiple outputs, what kind of things did you have in mind? I’d be happy to explore alternate outputs and see if I can do anything dynamic on the nodes based on their evaluation!

cheers :)

[MedleMedler]

Sure, will do! Let you know if I have a crash.

Good plan! In that case actually, the dream node would be multiple string inputs (and outputs) and multiple int, number and float inputs (or an option to choose). And like way like you have now: where inputs/outputs build out, when you connect one.

I see it like this: because of the "all round intelligence" of an ai chatbot, it is like magic in a box. Your Create agent node should be a "Stars Wars mothership", with endless ports to launch from and land on. In this case functionality creates opportunity and will create endless implementations

[shhlife]

So the agent does a run for every input, and has a resulting output per input - or based on their agents “work” it determines what the outputs should be?

Woild you be able to draw a graph of what you are thinking? It can be pretty rough - just to help me visualize it :)

[MedleMedler]

Good question. Like a Star Wars mothership, it should have independent inputs and outputs. That gives you more control. E.g. when you are satisfied with one output, you can freeze it and keep it that way. Also save time in calculation, the ai only calculates when it's needed when can freeze certain outputs.

Their are 4 kinds of input: string, number, integer and float. Number, integer and float, can have 1 input/output if have a choose option. In that choose option (scroll down menu) you can then. So that means that there are two types of input and output: string and number/ integer/float. Two kinds of inputs on the left side and two kinds of output on the right side.

Minimum 2/2 IN/OUT and as soon as you connect an in/output, the next in/output appears.

I think this will make compact multi functionality in one case.

Do you have any thoughts what else could be integrated?

[MedleMedler]

You can also put an freeze/activate option also in those little scroll down menus

[MedleMedler]

Something else: a text editor node would improve the usability significantly. I mean a text editor, for making text bold/color/italic/size. That way you have way more overview of e.g. your rule sets and organise your rules better with less mistakes.

A text editor node would also expand the kind use and users. It would also attract writers, who want to build writing workflows (with automatic notes, etc.,etc.).

[MedleMedler]

And all other basic text editor functionality, like saving in different formats

[MedleMedler]

I don't you need a visual now, or do you? Let me know, I am happy to make one.

I am also curious what your thoughts are?

[shhlife]

All are great ideas! A drawing would still be helpful - I want to make sure I’m imagining it the same way :)

some things (like the wysiwyg text editor) may be better handled as a default comfyui widget - I’ll look around and see if one exists.

in the meantime, keep the ideas coming! If you want to create new feature requests for each one that would help me keep track of them. :)

[MedleMedler]

Hermes told me that he has influence over its own temperature. He says that he can give temperature values to different topics. If this is true, then there should be multiple temperature fields/inputs in de config node.

I have made a rule, saying hermes that the temperature for output format, must always be 0.1., regardless if I make the minimum temperature higher.

Ideal would be, that you dynamically can set multi temperatures to diversify the temperature per topic (which you then tell the ai in the rule set)

[MedleMedler]

Is my sketch clear enough to understand? Left is with minimal in/outputs and as soon as you start connecting more in/outputs de node grows larger met more ports.

[MedleMedler]

Call this node Griptape Create: Agent Advanced and keep the old one for users that want a simple solution

[MedleMedler]

I do not know of it is possible, but putting the diversified temperatures (like a weight) in the little scroll down menus in the inputs (and/or outputs) will give much more control.

I think we have to try out and/or take some time to think this through.

[MedleMedler]

Idea: If you give Griptape Create: Agent Advanced node the same output functionality as Any Node, it will sure become a super Any Node. I am am not a programmer, but this could mean that there is a third kind of output; python code. But maybe this is not necessary, it depends if there are other nodes who need python code and/or you can use a string output for python code. I do not know how this works in Any Node, but it makes complex calculations with python and then writes a output. https://www.youtube.com/watch?v=Qn4h5z85vqw

I am now reconsidering the design of the new node. What if any input, can have any input format? The same with output. It would simplify the gui design of this node.

[MedleMedler]

For me it is still unclear, how the functionality and GripTape workflows work. The more I learn about GripTape nodes, the more I start to understand how power full it is, with it's hidden complexity. What would help is a huge library with workflow embedded images of Comfyui GripTape workflow (so you can upload the workflow, by dragging the image into Confyui. You need a showroom to show your Star Wars fleet and show your Star War war ships in how they combine.

This will boost the use, knowledge and use of more complex functionality, as well the creativity by users starting to play with it. This would GripTape so much more good in appreciation of users understanding the endless possibilities.

[MedleMedler]

Okay I slowly start to understand a setup With Create agent and Run agent. The point is/was that you can also use create agent as agent in itself. Up till now I did not use Run agent. I now see it like this: I have 1 Create agent and multiple Run agents (all directly connected to Create Agent), where every Run agents does his own task.

Concerning hermes3, I did manage to run a more complex test workflow with it.

Let me first understand the functionality of GripTape, before you start to make custom nodes.

[MedleMedler]

I am starting to get the hang of it. I now let one agent check the others for failures and if so, continually training them. This gives a huge boost in consistency. I think I actually made a loop, instructing hermes3 to do difficult parts double, while checking and training them in a feedback loop.

More an more, I start to get convinced that hermes3 can do complex tasks. The last few days I have seen him do very multiple complex tasks which he could autonomously handle, which totally amazed me. I wouldn't be surprised that hermes3 is better as specialised model then Chat Gtp 4o. I think hermes3 is very well suited for Comfyui.

Sometimes I really have to laugh, because hermes3 really has a character. He has a strong character, if he doesn't agree, then he will let himself hear in his suggestions. Much better then the general chat ai, who often talk you to the mouth.

[MedleMedler]

Seams like the FileManager is not working. I get a lot of these messages: <RetryCallState 1581230646416: attempt #1; slept for 0.0; last result: failed (DummyError You have attempted to use a DummyEmbeddingDriver's try_embed_chunk method. This likely originated from using a DriversConfig without providing a Driver required for this feature.)>

[MedleMedler]

I want to try Qwen2-Math, to see if hermes3 is the problem. But I now get this message: Error occurred when executing Griptape Create: Agent:

The api_key client option must be set either by passing api_key to the client or by setting the OPENAI_API_KEY environment variable

I now tried hermes out a few days, and the first day it went like a charm. After that, so now and then I had successes. But a lot of times it could not find the FileManager.

[shhlife]

Oh that's an interesting one - I'll look into it. It's probably a simple config issue. I'm out today, but will look at it tomorrow!

[MedleMedler]

And now get also an error (after changing to Qwen2-Math after the API key message), that hermes is also blocked by API key. And you can't select any model in Griptape Agent Config: Ollama. Only solution is remove this node and connect a new one.

[MedleMedler]

After I downloaded Qwen2-Math (through the desktop app Ollama and de windows command shell), I also get API Key error message. Downloading Qwen2-Math now caused a permanent API Key problem with all open source models. I will now delete Qwen2-Math and will look if the API Key problem will go away.

In the meantime, I will wait for you, to look into it.

Strange enough, I hope all problems are related to the nodes, because those are solvable. It still could well be, the ai is the problem. If that's the case, then it will be a matter of weeks (few months), that new ai models can.

[MedleMedler]

After deleting Qwen2-Math, the API KEY problem stays.

[shhlife]

I’ll check it out tomorrow and see if I can fix it. Can you send a screenshot of your node layout? That’ll help me reproduce it :)

On Mon, 2 Sep 2024 at 5:08 PM, MedleMedler @.***> wrote:

After deleting Qwen2-Math, the API KEY problem stays.

— Reply to this email directly, view it on GitHub https://github.com/griptape-ai/ComfyUI-Griptape/issues/82#issuecomment-2323824078, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABJFWYMW6NSWFR4KLAF3EITZUPXDRAVCNFSM6AAAAABLGWPYA2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMRTHAZDIMBXHA . You are receiving this because you modified the open/close state.Message ID: @.***>

[MedleMedler]

Thank you very much!

I now even uninstalled Griptape, restarted my pc and installed Griptape again and the API error still stays: Error occurred when executing Griptape Create: Agent:

The api_key client option must be set either by passing api_key to the client or by setting the OPENAI_API_KEY environment variable

File "D:\ComfyUI_windows_portable_nvidia\ComfyUI\execution.py", line 317, in execute output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_windows_portable_nvidia\ComfyUI\execution.py", line 192, in get_output_data return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_windows_portable_nvidia\ComfyUI\execution.py", line 169, in _map_node_over_list process_inputs(input_dict, i) File "D:\ComfyUI_windows_portable_nvidia\ComfyUI\execution.py", line 158, in process_inputs results.append(getattr(obj, func)(**inputs)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_windows_portable_nvidia\ComfyUI\custom_nodes\ComfyUI-Griptape\nodes\agent\BaseAgent.py", line 146, in run create_dict["tools"] = self.tool_check(config, tools) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_windows_portable_nvidia\ComfyUI\custom_nodes\ComfyUI-Griptape\nodes\agent\BaseAgent.py", line 124, in tool_check tools.append(QueryTool(off_prompt=False)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "", line 28, in init _setattr('prompt_driver', attr_factory_prompt_driver()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_windows_portable_nvidia\python_embeded\Lib\site-packages\griptape\tools\query\tool.py", line 28, in prompt_driver: BasePromptDriver = field(default=Factory(lambda: Defaults.drivers_config.prompt_driver)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_windows_portable_nvidia\python_embeded\Lib\site-packages\griptape\utils\decorators.py", line 42, in lazy_attr setattr(self, actual_attr_name, func(self)) ^^^^^^^^^^ File "D:\ComfyUI_windows_portable_nvidia\python_embeded\Lib\site-packages\griptape\configs\drivers\openai_drivers_config.py", line 20, in prompt_driver return OpenAiChatPromptDriver(model="gpt-4o") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "", line 19, in init self.client = attr_factory_client(self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_windows_portable_nvidia\python_embeded\Lib\site-packages\griptape\drivers\prompt\openai_chat_prompt_driver.py", line 60, in lambda self: openai.OpenAI(api_key=self.api_key, base_url=self.base_url, organization=self.organization), ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_windows_portable_nvidia\python_embeded\Lib\site-packages\openai_client.py", line 98, in init raise OpenAIError( Queue size: 0 CPU 1% RAM 10% GPU 0 0% VRAM0 3% Temp 0 38° GPU 1 24% VRAM1 13% Temp 1 43° HDD 82% ERROR Extra options

[MedleMedler]

Anyway: searching for alternatives, I found out that there other LLM loaders with more settings, which are useful for better control of the ai model.

Top-p (top_p): Also known as nucleus sampling, this parameter controls the cumulative probability distribution cutoff. The model will only consider the top p% of tokens with the highest probabilities for sampling. Reducing this value helps in controlling the generation quality by avoiding low-probability tokens. Default: 0.9.

Top-k (top_k): Limits the number of highest probability tokens considered for each step of the generation. A value of 0 means no limit. This parameter can prevent the model from focusing too narrowly on the top choices, promoting diversity in the generated text. Default: 50.

Repetition Penalty (repetition_penalty): Adjusts the likelihood of tokens that have already appeared in the output, discouraging repetition. Values greater than 1 penalize tokens that have been used, making them less likely to appear again. Default: 1.2.

Trust Remote Code (trust_remote_code): A security parameter that allows or prevents the execution of remote code within loaded models. It is crucial for safely using models from untrusted or unknown sources. Setting this to True may introduce security risks. Default: False.

Torch Data Type (torch_dtype): Specifies the tensor data type for calculations within the model. Options include "float32", "bfloat16", "float16", "float64", or "auto" for automatic selection based on device capabilities. Using "bfloat16" or "float16" can significantly reduce memory usage and increase computation speed on compatible hardware. Default: "auto".

I think that integrating these settings in the GripTape config nodes, will improve the output quality. Not sure if all these settings are useful and if there are other settings which will help to fine tune the ai model more, for better, more/less consistent results.

[MedleMedler]

Here are a few other settings

[MedleMedler]

I am actually very happy that I got stuck. I found other probably more interesting models. Fine tunes of Hermes 3. Like this one: https://huggingface.co/vicgalle/Configurable-Hermes-3-Llama-3.1-8B

Which is configurable through a json file. I can use slider nodes to set these setting, merging them like below and export this as json file. So I can adjust it in comfyui. { "instruction-following-threshold": 0.5, "instruction-following-max-steps": 10, "question-answering-threshold": 0.7, "question-answering-max-steps": 15, "text-generation-threshold": 0.9, "text-generation-max-steps": 20 }

Do you have a config node, where you can upload this model (from a folder location in Comfyui)?

[MedleMedler]

OK somehow suddenly Qwen2-math is now running. Looks like it was the set default agent, now I have connected it correctly. Id o not know why Griptape did run without connecting the set default agent through. Hope this will solve all my issues and can se hermes/qwen2 do his magic consistenly again.

Quite a bumpy ride, but I learned a lot searching for another solution

[MedleMedler]

I promised you feedback, I now have delivered for sure, lol

[MedleMedler]

Maximum attemps on fail does not work, if you lower it down

[MedleMedler]

Qwen2-math can get acces to filemanager. Hermes can again. I will keep testing it on failures

[shhlife]

ah, I figured out this morning why you were getting the requirement for the OpenAI API Key - whenever you use tools that has off_prompt=True, there response from the tool is automatically stored in a local vector memory, keeping it out of the LLM. Then, Griptape will use something called a QueryTool to do a semantic search on the memory to find the answer - this allows you to do things like not scrape an entire website & send it to the LLM.

In order to do a semantic search, the agent needs to have a VectorEmbeddingDriver to know how to handle the embeddings.

The default is to use OpenAi's Vector Embedding driver - which requires an API key.

But you can work your way around this by creating the Config yourself using the Griptape Agent Config: Custom Structure node where you specify both the prompt_driver and embedding_driver. Here, I've set the embedding driver to one for for ollama: nomic-embed-text (https://ollama.com/library/nomic-embed-text).

Now the agent won't require the OpenAI API Key.

To make this clearer, I'll update the Griptape Agent Config node to include embedding drivers, since they're often required for tool use.

If you really don't want to include an embedding driver for some reason, you can also just set off_prompt to False for the fileManager tool:

[shhlife]

btw - I'm working on changing the configuration nodes to make them more complete - for example, here's the new OpenAI one:

This should hopefully make it much clearer what drivers each model is using