Improve Pro User Model Selection

[chillatom]

Problems

• Model selection drop down list is long
• No guidance to user on when to choose a particular model
• Mix of legacy, new and experimental models

Desired state

• Clear guidance on model choice (Good, Better, Best, Fast, Faster, Fastest etc.)
• Legacy models deprioritized in rank and clearly annotated (e.g. Claude 2.0, Claude 2.1)
• Experimental models deprioritized in rank and clearly annotated (e.g. Mixtral models)

[philipp-spiess]

I’m sure that @toolmantim has better ideas but ever since we chatted about surfacing the differences of models, I've been thinking of this as a 3 point scale across two dimensions and have envisioned a UI like this:

[toolmantim]

I took a stab at a first draft of this here: https://www.figma.com/file/jEsnzgsf0hNuJbqB9pKaGm/VS-Code---Cody---Commands-up-front-and-ChatGPT%2B%2B?type=design&node-id=2270-46492&mode=design&t=PP9VQP2xJha34ErH-4

Click on one of the model-selector frames and press shift-space to prototype the hover/scroll. See the comments for notes.

Let me know what you think!

[toolmantim]

Nice @philipp-spiess — love this sketch! I ended up going with a simpler grouping… but you know what, we could actually make the selector an actual 2-axis chart that you use to select the model. As long as you make it clear which ones are the recommended ones.

Not sure if we've got enough data/info to be that granular to compare model quality/speed though, do we? And it adds a bit of a bar for adding a model… vs just having an "Experimental" box to throw stuff into w/o clear data.

OR going back to something simpler… we just add some little indicators/icons to the right of each row. Only if we feel it's really needed though.

[philipp-spiess]

@toolmantim Ha love the idea but the route you're going in the Figma is probably enough for now and already a good improvement (and much easier to visually parse).

[abeatrix]

I love the grouping idea!

Personally think it'd be helpful to show what tasks each model is good at to help users to decide which model to use! Something like: Claude Opus: Great for answering large codebase questions, or Claude Haiku for code formatting etc

Ref: GPT marks ChatGPT3.5 as Great for everyday tasks and ChatGPT4 as the smallest and most capable 😆

A + button to add new modeld would be cool too like they do in Opera:

[philipp-spiess]

Love the plus button idea! Could link to some page explaining ollama etc :) Not sure about the text from ChatGPT. I think this UI only works if you have very few models. Imagine presenting 6 models like this. You'd have to scroll pretty quickly. Plus: How would you describe GPT4 vs Opus in these terms? The differences become blurry quickly.

[sqs]

Model selection should be for free users, too, but we can still gate certain expensive models.

[toolmantim]

Updated design in Slack: https://sourcegraph.slack.com/archives/C04MSD3DP5L/p1713858613785279