Brave Leo: Spinning wheel (with apologies to BST)

[mrose17]

cc: @petemill, @mattmcalister

Description

I am moving https://github.com/brave/aichat/issues/126 to this repo.

This is an issue with too much text for Mixtral (and probably llama, though I didn't test that.

This issue is not about Mixtral, per se, it's about how we handle text that is larger than the buffer size. The basic issues are:

• "New chat" doesn't appear to reset anything
• "Suggest questions ..." gets a spinning wheel that never goes away (until you close the tab)
• Switching to a model with a larger buffer will give back answers (with a spinning wheel)
• Some times after "This conversation is too long" after clicking on "New chat" you get a "network issue connecting to Leo".

Steps to Reproduce

Start here:

Click on 'Summarize this page' and get this:

The text with the New chat button usually appears first, followed a moment later the text with the 'Got it`button appears. But sometimes, the order is switched. This doesn't matter, it's just a bit odd.

Click on 'Got it'.

Click on 'New chat' to get this:

Click on 'Suggest questions..." and you get the spinning wheel here:

The wheel does not go away.

Switch to Claude Instant and get this:

Enter "Summariz+e this page." and it works for Claude Instant. However the wheel is still spinning.

However, if you switch back to Mixtral and enter" Summarize this page." you get this:

Click on "New Chat" and you get this:

Click on "Retry" and you get this:

The wheel keeps on spinning too!

Expected result:

1. No spinning whell
2. "New Chat" clears the state.

Reproduces how often:

100%

Brave version (brave://version info)

I am running Version 1.64.5 Chromium: 121.0.6167.75 (Official Build) nightly (arm64) on an MBA with Sonoma 14.2.1 (the latest).

Version/Channel Information:

Only Nightly tested.

• Can you reproduce this issue with the current release? N/A
• Can you reproduce this issue with the beta channel? N/A
• Can you reproduce this issue with the nightly channel? Yes.

Other Additional Information:

• Does the issue resolve itself when disabling Brave Shields? N/A
• Does the issue resolve itself when disabling Brave Rewards? N/A
• Is the issue reproducible on the latest version of Chrome? N/A

[mrose17]

Can't reproduce and no one else has seen it.

[mrose17]

It hit me again. STR:

1. Use Claude Instant.
2. Have a 25+K transcript.
3. Claude Instant works fine.
4. Switch to Llama.
5. Ask a question and it will warn about the text size, but answer.
6. Switch to Mixtral.
7. It will warn about the text size and not answer.
8. At this point, no more prompts are processed.
9. I switched back to Claude Instant, no luck.
10. I cleared the chat, no luck: