Getting error: "The given key was not present in the dictionary"

[PreDawnHydra]

I am getting this error when I ping the bot. Would this be related to the startup args for oobabooga's text generation webui? My current args are: --model TheBloke_airoboros-l2-7B-gpt4-m2.0-GPTQ --extensions api --loader exllama --listen-port 5000 --wbits 4 --groupsize 128

I put the args in the "CMD_FLAGS.txt"

I also noticed the above warning but the bot seems to start up fine its when I ping the bot it throws the error, but the warning could be related in some way?

Thanks!

[DeSinc]

ahh this first error in the CMD screenshot you sent, I used to get that while I was developing the bot and it was reading the json format incorrectly. are you using an old version of sallybot or oobabooga text-gen-webui* by chance?

Just note, if you update text-gen-webui, as of last month, you need to download this snapshot of it, go into the extensions folder, and pull out the 'api' folder and replace your current text-gen-webui's 'api' folder with that one. (You can rename the existing 'api' folder to something else and keep it there if you want to be able to use it again later.)

https://github.com/oobabooga/text-generation-webui/archive/refs/tags/snapshot-2023-11-12.zip ^ That is oobabooga's text-gen-webui snapshot with the old API files inside the extensions folder.

The reason you have to do this is because they updated to use OpenAI's API format and deprecated (totally removed) the old API that I built sallybot on, so you need to do those steps to get the old API back.

(my edit accidentally closed the issue somehow? I don't even know how that happened)

[PreDawnHydra]

Gotcha yep using the old API seems to have fixed it the bot now responds to prompts!

The only other thing is when I request an image to be generated it says "Request Failed:" without any further info

Stable diffusion is running on port 7860 the bot is also set to the same port in the Functions.cs

[DeSinc]

It's possible I've updated my local bot to fix this issue and forgotten to carry over the change to the github. (not 100% sure though)

I will investigate

[DeSinc]

I can't see any differences between my local code and the github code so it should work. Are you using the --api arg in the SD launch args?

[PreDawnHydra]

No problem thanks for taking a look! I'll be away from my computer for a while but I'll double check everything tonight! I think I only have the --extensions api not the --api arg

[DeSinc]

Yeah --extensions api is for text-gen-webui, text not pictures

You need --api in the Stable Diffusion launch args to generate pictures

I mix them up all the time, just clarifying to be sure

[PreDawnHydra]

Yep adding --api to SD seems to have fixed everything, the bot sends images now!

I noticed depending on how long Stable Diffusion generation takes it looks like the request can timeout sometimes (after 100 seconds), do I need to change that somewhere?

Sry ik this is really off topic from the original issue at this point. Anyway this issue should be good to close just figuring out how to configure everything!

[DeSinc]

The 100 second timeout, yeah I get that too and I'm not 100% sure where to fix that. It's probably an HTTP request timeout, which I'm finding a hard time changing for some reason. Everything online just says to use the ".Timeout" or "Options.MaxTimeout" value but it just keeps saying it's read only or doesn't exist so I'm figuring out what I've probably done differently to them.

Good to see the main issues are sorted now at least :)

[DeSinc]

Oh yeah I totally forgot to tell you, I figured out how to fix the timeout and pushed an update to the bot a few hours ago so if you pull the update you will have it automatically. I set the timeout to be 200 seconds now so that should give more time to generate images, but you can increase this time more if you change the 'MaxTimeout' and 'sdImgRequest.Timeout' sections in the TakeAPic() code block in Functions.cs.

I also included like nearly a year's worth of updates so that should be fun for you. The bot now polls for chat updates and asks itself if it wants to reply on its own accord, on top of just responding to straight pings and mentions. There's also now Google API support (the gemini pro model API is free indefinitely for up to a max of 60 requests every minute would you believe) and async messaging so the bot doesn't freeze while it generates a reply or an image now.

[PreDawnHydra]

That's awesome! Thanks! :D

Haven't worked with Google's gemini API stuff before but I went ahead and generated an API key and added it to the new field though to test out later. After adding it to the field in MainGlobal.cs is there anything else I need to do to enable/set it up?

[DeSinc]

In order to turn on Gemini I made it so you have to type a specific enable phrase to prevent it using the API without your express knowledge. To enable it you have to be the server owner of the discord server and type enable gemini in the chat, then the bot will switch over to using the API.

I find the Gemini model is quite good, probably smarter or more practically useful in terms of reasoning than any open source model with the possible exception of mixtral 8 x 7b which seems pretty smart but takes up like 24GB of vram to run well. The only downside is the model has some built-in guardrails so if you shout at the bot too much it interprets it as harassment and it won't give you a response, etc. it's also not the most natural chatter, so it sounds a little robotic as most models do, not like the good RP models or airoboros llama 2, which I consider excellent at natural chatty flavour.