imtuyethan commented 3 months ago

Problem

When user delete a model & go back to a specific threads that uses the deleted model. E.g. Thread A uses model Llama 3 & user deletes model Llama 3 then go back to thread A & continue the chat...

Current behavior: We switch to another model without user's permission, which is not cool.
Expected behavior: We should inform user that the selected model is deleted & recommend them to download or select another model.

Or:

When Remote model is unreachable (e.g. if offline). Current behavior: We only show error Failed to fetch upon user sending message.

Design

Scenario 1: The selected `on-device` model is unavailable

E.g., user deleted that model

Figma: https://www.figma.com/design/DYfpMhf8qiSReKvYooBgDV/Jan-App-(3rd-version)?node-id=8322-71957&t=Eu48etLJ7eAZ5FDp-4

0. User has a thread that they created previously; then, they delete the model used in that thread

1. User opens that thread again

2. System shows error indicator in Model Selector Field:

3. User can click on the Model Selector to choose another model:

Users can see that the model used (Mistral Instruct 7B Q4) has not been downloaded

4. If user does not change the model & attempt to send a message, it will show error:

This error is needed because there are cases when users collapse all panels & they don't see error indicators:

5. Chat continues normally after new model selection

Scenario 2: The selected `remote` model is unavailable

E.g., when there's no connection

Figma link: https://www.figma.com/design/DYfpMhf8qiSReKvYooBgDV/Jan-App-(3rd-version)?node-id=8984-46951&t=hNAHVWiL94qZVMpx-4

1. Similarly, system shows error indicator in Model Selector Field:

2. User can click on the Model Selector to choose another model:

User can see an indicator at the bottom that No internet connection. Please use on-device models

4. If user does not change the model & attempt to send a message, it will show error:

Tasklist

[x] Designs @imtuyethan
[ ] UI Implementation @urmauur

Related Issues

3537

imtuyethan commented 2 months ago

Discord discussion: https://discord.com/channels/1107178041848909847/1273597436920594466

dan-homebrew commented 2 months ago

I think there are two potential scenarios here:

Model has been deleted (or doesn't work) => choose new model
All models and Remote APIs have been deleted

My view is that we should prompt the user to choose, instead of just using "next model". That way, we can solve for both situations at once, and not add an unnecessary edge case:

Scenario 1: Thread model is deleted

Specific Thread's model is deleted
e.g. llama3 is deleted, and all 30 threads that use it don't have a model any more
Our response: "The previous model is no longer available. Please choose a new default model" -> Selector

imtuyethan commented 2 months ago

Dup https://github.com/janhq/jan/issues/3537

dan-homebrew commented 2 months ago

@imtuyethan I am breaking up our original issue into two issues:

This issue
3537 (covers all model deleted case)

Can you reply to this issue with the Wireframes (for this specific case).

I am also queueing this in Sprint 20, as @urmauur may be able to get to it next week.

imtuyethan commented 2 months ago

Archive Original Issue

### Case 1: Users delete model of a thread
In this scenario, a user has deleted or removed the specific model associated with an existing conversation thread. The interface shows:
- The model dropdown now displays "Mistral Instruct 7B Q4" but is grayed out, indicating it's no longer available
- An error message in the chat area states "Current model is not available. Please download or choose a different one."

![Screenshot 2024-09-09 at 9 44 40 AM](https://github.com/user-attachments/assets/b8d5aaa3-34f5-4c02-a9c2-75896fdbabfd)
https://www.figma.com/design/DYfpMhf8qiSReKvYooBgDV/Jan-App-(3rd-version)?node-id=8322-71957&t=EerQsMye51GlIUlB-4

### Case 2: Users delete all models or API key but continue using old threads
In this case, the user has deleted all local models or removed their API key, but is attempting to continue using existing conversation threads. The interface shows:
- The model dropdown displays "Select a model" with no options available
- An error message in the chat area states "You don't have any available model." with a "Go to Hub" button

![Screenshot 2024-09-09 at 9 47 03 AM](https://github.com/user-attachments/assets/f7e99713-f7c0-46fd-9fd8-b833c681bef2)
https://www.figma.com/design/DYfpMhf8qiSReKvYooBgDV/Jan-App-(3rd-version)?node-id=8124-71826&t=EerQsMye51GlIUlB-4

imtuyethan commented 2 months ago

^ Updated the description with new design @dan-homebrew

dan-homebrew commented 2 months ago

@imtuyethan Quick feedback:

Is there a visual indicator we can use to prompt user that they need to change the model?

Currently, it's greyed out, with the old model name
Grey out = won't user think it's disabled?
Should we draw their attention to the thing that needs changing, e.g. with a glow around the model selector?

I also think we can simplify the copywriting for the error message:

https://www.nngroup.com/articles/ten-usability-heuristics/

imtuyethan commented 1 month ago

Updated design: https://www.figma.com/design/DYfpMhf8qiSReKvYooBgDV/Jan-App-(3rd-version)?node-id=8322-71957&t=t7FOVtMEOSg7nFsJ-4

cc @dan-homebrew

imtuyethan commented 1 week ago

User flows & scenarios

Scenario 1: The selected `on-device` model is unavailable

E.g., user deleted that model

Figma: https://www.figma.com/design/DYfpMhf8qiSReKvYooBgDV/Jan-App-(3rd-version)?node-id=8322-71957&t=Eu48etLJ7eAZ5FDp-4

0. User has a thread that they created previously; then, they delete the model used in that thread

1. User opens that thread again

2. System shows error indicator in Model Selector Field:

3. User can click on the Model Selector to choose another model:

Users can see that the model used (Mistral Instruct 7B Q4) has not been downloaded

4. If user does not change the model & attempt to send a message, it will show error:

This error is needed because there are cases when users collapse all panels & they don't see error indicators:

5. Chat continues normally after new model selection

Scenario 2: The selected `remote` model is unavailable

E.g., when there's no connection

Figma link: https://www.figma.com/design/DYfpMhf8qiSReKvYooBgDV/Jan-App-(3rd-version)?node-id=8984-46951&t=hNAHVWiL94qZVMpx-4

1. Similarly, system shows error indicator in Model Selector Field:

2. User can click on the Model Selector to choose another model:

User can see an indicator at the bottom that No internet connection. Please use on-device models

janhq / jan

feat: Pick new model in thread if original model is unavailable #3385

Problem

Design

Scenario 1: The selected `on-device` model is unavailable

0. User has a thread that they created previously; then, they delete the model used in that thread

1. User opens that thread again

2. System shows error indicator in Model Selector Field:

3. User can click on the Model Selector to choose another model:

4. If user does not change the model & attempt to send a message, it will show error:

5. Chat continues normally after new model selection

Scenario 2: The selected `remote` model is unavailable

1. Similarly, system shows error indicator in Model Selector Field:

2. User can click on the Model Selector to choose another model:

4. If user does not change the model & attempt to send a message, it will show error:

Tasklist

Related Issues

3537

Scenario 1: Thread model is deleted

3537 (covers all model deleted case)

Archive Original Issue

User flows & scenarios

Scenario 1: The selected `on-device` model is unavailable

0. User has a thread that they created previously; then, they delete the model used in that thread

1. User opens that thread again

2. System shows error indicator in Model Selector Field:

3. User can click on the Model Selector to choose another model:

4. If user does not change the model & attempt to send a message, it will show error:

5. Chat continues normally after new model selection

Scenario 2: The selected `remote` model is unavailable

1. Similarly, system shows error indicator in Model Selector Field:

2. User can click on the Model Selector to choose another model:

4. If user does not change the model & attempt to send a message, it will show error:

janhq / jan

feat: Pick new model in thread if original model is unavailable #3385

Problem

Design

Scenario 1: The selected on-device model is unavailable

0. User has a thread that they created previously; then, they delete the model used in that thread

1. User opens that thread again

2. System shows error indicator in Model Selector Field:

3. User can click on the Model Selector to choose another model:

4. If user does not change the model & attempt to send a message, it will show error:

5. Chat continues normally after new model selection

Scenario 2: The selected remote model is unavailable

1. Similarly, system shows error indicator in Model Selector Field:

2. User can click on the Model Selector to choose another model:

4. If user does not change the model & attempt to send a message, it will show error:

Tasklist

Related Issues

3537

Scenario 1: Thread model is deleted

3537 (covers all model deleted case)

Archive Original Issue

User flows & scenarios

Scenario 1: The selected on-device model is unavailable

0. User has a thread that they created previously; then, they delete the model used in that thread

1. User opens that thread again

2. System shows error indicator in Model Selector Field:

3. User can click on the Model Selector to choose another model:

4. If user does not change the model & attempt to send a message, it will show error:

5. Chat continues normally after new model selection

Scenario 2: The selected remote model is unavailable

1. Similarly, system shows error indicator in Model Selector Field:

2. User can click on the Model Selector to choose another model:

4. If user does not change the model & attempt to send a message, it will show error:

Scenario 1: The selected `on-device` model is unavailable

Scenario 2: The selected `remote` model is unavailable

Scenario 1: The selected `on-device` model is unavailable

Scenario 2: The selected `remote` model is unavailable