Is your feature request related to a problem? Please describe it
Currently, Jan only offers four predefined OpenAI models, which may not reflect the full range of models available to users with different API key access levels. This limitation prevents users from accessing their full range of accessible models, including older, more powerful models.
Describe the solution
I would recommend implementing a dynamic model discovery feature that runs every time the user edits their OpenAI API key that queries the OpenAI API to retrieve all available models for the new key.
Maybe you update the AIEngine class (in core/src/browser/extensions/engines/AIEngine.ts:35-59) to incorporate this dynamic model discovery
Definitely add a new field to the model object in models.json to track whether a model is enabled (extensions/inference-openai-extension/resources/models.json): {"enabled": true}
To account for the possible drawback:
Update the UI to include a toggle for each model in the OpenAI settings, allowing users to enable/disable models (optional, to account for the drawback mentioned in issue 3115).
Modify the model loading process (core/src/node/api/restful/helper/builder.ts:300-316) to filter out disabled models.
Potential Drawbacks
Handling the case of no internet when the API key is changed to fail gracefully
Managing updates when OpenAI introduces or deprecates models (actually, this helps with the case of adding new models)
If we don't include the configurable model options, this could add more clutter by adding more API "models" to the list
Enable the Feature:
Navigate to the settings page.
Enable/disable the "Dynamic Model Discovery" option (should be enabled by default).
Configure Models:
Go to Settings ⌘, > OpenAI > "Model Configuration" section (this is a new section that needs to be added)
Toggle the models you want to enable or disable.
What is the motivation / use case for changing the behavior?
Use gpt-4-0314, currently the smartest, least restricted model available, to answer questions on a local machine not powerful enough to run high-end models locally.
0xSage
commented
1 month ago
Is your feature request related to a problem? Please describe it
Currently, Jan only offers four predefined OpenAI models, which may not reflect the full range of models available to users with different API key access levels. This limitation prevents users from accessing their full range of accessible models, including older, more powerful models.
Describe the solution
I would recommend implementing a dynamic model discovery feature that runs every time the user edits their OpenAI API key that queries the OpenAI API to retrieve all available models for the new key. Maybe you update the
AIEngineclass (incore/src/browser/extensions/engines/AIEngine.ts:35-59) to incorporate this dynamic model discovery Definitely add a new field to the model object inmodels.jsonto track whether a model is enabled (extensions/inference-openai-extension/resources/models.json):{"enabled": true}To account for the possible drawback: Update the UI to include a toggle for each model in the OpenAI settings, allowing users to enable/disable models (optional, to account for the drawback mentioned in issue 3115). Modify the model loading process (
core/src/node/api/restful/helper/builder.ts:300-316) to filter out disabled models.Potential Drawbacks
Teachability, documentation, adoption, migration strategy
Teachability: The users won't even notice.
For docs:
Enable the Feature: Navigate to the settings page. Enable/disable the "Dynamic Model Discovery" option (should be enabled by default). Configure Models: Go to Settings
⌘,> OpenAI > "Model Configuration" section (this is a new section that needs to be added) Toggle the models you want to enable or disable.What is the motivation / use case for changing the behavior?
Use
gpt-4-0314, currently the smartest, least restricted model available, to answer questions on a local machine not powerful enough to run high-end models locally.