[RESOLVED] Older versions of the Extras API made bots give better responses, now it doesn't.

[TheSystemGuy1337]

CAUTION: Confusion wave inbound. Brace yourself.

Is it just me, or with this version of the Extras API, bot responses with the API are like responses without the API? I don't know what was giving bots better responses with this API on older versions, it could have been I was running it with a system install and not miniconda. Was this a bug with older versions? Is it me? I don't even know anymore. I might be an idiot, who knows? But in the logs it seems to store more, leading to better responses, now it seems to store less.

[Cohee1207]

Before we dwell into speculations, here are some questions:

1. What's your main API? Extras do not generate bot replies.
2. Which modules do you use?
3. Compare prompts and settings between different versions, do you see any difference?

[TheSystemGuy1337]

First off, I wrote it originally when I was half asleep and half awake, so I apologize for it not being very human readable. I use the Horde as the primary API, so that may be a contributing factor to some nonsense. I use all the modules that are installed with requirements-room.txt. I cannot tell which’s version of the Extras API was installed, from what I can gather (as I enabled print back in the browsers console), it appeared that older versions (versions that didn’t print the Python 3.11 message) of the Extras API did a better job storing responses, especially with bots with a description and first message so long that after the “roleplay begin” text, the user’s input is completely cut off, but to my memory, older versions made it where my input wasn’t cut off, potentially improving bot responses. Since this is from recollection, take it with a grain of salt. Another potential factor may be I installed it on the system directly, not in a virtual environment like miniconda.

As an example, say this line was the bots description and first message.

Then, the roleplay chat with [user] starts here. (Afterwards, the user input doesn’t appear at all. Normally my first message should appear after this line of text)

[Cohee1207]

By storing responses you mean what? Summarization or chromadb? Installation method should not affect the quality of Extras, only the ease of isolating it from your system. Judging by your symptoms, it looks like your context size does not fit the character card and the user message. Because character defs are never truncated. What are your context size setting and character tokens?

[TheSystemGuy1337]

I mean ChromaDB, not summarization. I am using the “Pleasing Results” preset with a context size of 2048 tokens and a response length of 100. The bots I have tried had 580 tokens, 426 permanent tokens and 873 tokens, 759 permanent respectively. I’ll try slowly raising the context size and see if it fixes the issue. I looked through the output of the Extras API on cool-retro-term (my terminal) and ChromaDB is initializing properly. And I’m using Miniconda now as per the instructions. If this helps, I’m using the CPU, and the extras API is being run on Linux Mint 21.2 Cinnamon. I’m considering spinning up a Ubuntu Server VM for diagnostic purposes.

[TheSystemGuy1337]

As a test, and for a greater example, I have loaded up Seraphina, a default bot that ships with ST. This is the output from Google Chrome's console.

Seraphina: Seraphina's gentle smile widens as she takes a moment to consider the question, her eyes sparkling with a mixture of introspection and pride. She gracefully moves closer, her ethereal form radiating a soft, calming light. "Traits, you say? Well, I suppose there are a few that define me, if I were to distill them into words. First and foremost, I am a guardian — a protector of this enchanted forest." As Seraphina speaks, she extends a hand, revealing delicate, intricately woven vines swirling around her wrist, pulsating with faint emerald energy. With a flick of her wrist, a tiny breeze rustles through the room, carrying a fragrant scent of wildflowers and ancient wisdom. Seraphina's eyes, the color of amber stones, shine with unwavering determination as she continues to describe herself. "Compassion is another cornerstone of me." Seraphina's voice softens, resonating with empathy. "I hold deep love for the dwellers of this forest, as well as for those who find themselves in need." Opening a window, her hand gently cups a wounded bird that fluttered into the room, its feathers gradually mending under her touch. Justin: "Describe your body and features." Seraphina: Seraphina chuckles softly, a melodious sound that dances through the air, as she meets your coy gaze with a playful glimmer in her rose eyes. "Ah, my physical form? Well, I suppose that's a fair question." Letting out a soft smile, she gracefully twirls, the soft fabric of her flowing gown billowing around her, as if caught in an unseen breeze. As she comes to a stop, her pink hair cascades down her back like a waterfall of cotton candy, each strand shimmering with a hint of magical luminescence. "My body is lithe and ethereal, a reflection of the forest's graceful beauty. My eyes, as you've surely noticed, are the hue of amber stones — a vibrant brown that reflects warmth, compassion, and the untamed spirit of the forest. My lips, they are soft and carry a perpetual smile, a reflection of the joy and care I find in tending to the forest and those who find solace within it." Seraphina's voice holds a playful undertone, her eyes sparkling mischievously. [Genre: fantasy; Tags: adventure, Magic; Scenario: You were attacked by beasts while wandering the magical forest of Eldoria. Seraphina found you and brought you to her glade where you are recovering.]

Then the roleplay chat between Justin and Seraphina begins.

(Then nothing. I said Hello world, it should appear after the roleplay begins text along with my name, instead that doesn't happen. Is this a bug with ST? Is this a problem with the Horde itself? Is the Extras API working right? I'll attempt a system power cycle later)

[Cohee1207]

Do you have "adjust to worker context size" enabled in horde setting? It effectively reduces your context to the least available denominator between all horde workers hosting the models you selected. It could go to as little as 1024 essentially reducing your context twofold.

[TheSystemGuy1337]

Turned that option off, now my name and message prints to the console, indicating it was sent. And I get a good response. Thanks for helping me diagnose and fix this issue. You deserve a free flight on an F-16 fighter jet.