Configuring default LLM for chat window

[ssp24]

I have been able to set a default llm and export an API-key to be able to use straight away when starting my jupyter lab. But it seems that I can only access this through the magic commands. What I'd really like to do is configure Jupyter AI in a way that when I start Jupyter Lab and click on the little Jupyter AI Chat window, the same LLM is already preselected from the list and I don't have to select anything first but can start using the chat window straight away with the LLM already set as a default. My understanding is that this isn't possible so far, is that right? If so, that would be a great feature!

[JasonWeill]

@ssp24 The magic commands are configured differently from the chat window. In the chat panel, when you go into the settings, you should be able to select a language model and provide an API key for it, then save your settings. These settings should persist across sessions. When you've configured the chat UI in this way, do you see the selected LLM persist after you close and restart JupyterLab?

[JasonWeill]

Also note that #933 was a bug involving saving settings — this was just fixed, and hasn't been incorporated into a release yet.

[dlqqq]

@ssp24 We do support having a default language model. Basically, you just need to pass the --AiExtension.default_language_model argument when starting JupyterLab. For example:

jupyterlab --AiExtension.default_language_model=anthropic-chat:claude-2.1

When JupyterLab opens, Jupyter AI should default to anthropic-chat:claude-2.1. Note that this only works when the user did not select a model and save it to their settings previously; we always respect the user's saved settings if they exist.

Check out our documentation for more details on the configuration options we offer: https://jupyter-ai.readthedocs.io/en/latest/users/index.html#configuring-default-models-and-api-keys

Hope this helps!

[ssp24]

Thanks for the clarification. I was hoping there would or will be way to set a default for the chat window as well, even before I as a user have selected a model. My use case being that I want to push a preconfigured environment with jupyter-ai to binder and make it accessible for a group of users who should just be able to "chat away for help" in the chat window as well without having to worry about selecting a model and API-keys first. Is there currently a way of achieving this, maybe with a workaround?

[krassowski]

I was hoping there would or will be way to set a default for the chat window as well, even before I as a user have selected a model

I do not follow, this is exactly what --AiExtension.default_language_model does.

[ssp24]

Hm, that hasn't worked for me so far, what happend was that the default LLM was selected and I was able to use the magic commands in a code cell straight away, but when I clicked on the chat window I still got the "empty" version where I had to select an LLM myself. You're saying this should have been different? Will try it out again to see if I missed something

[ssp24]

Ok I was able to try this again and now it is working, which is great! I am still having two issues though and am hoping you might be able to help:

I want to pass a default LLM as well as a corresponding API key at the same time when starting jupyter lab. I've tried to simply combine them via jupyter lab --AiExtension.default_language_model=openai-chat:gpt-4 --AiExtension.default_api_keys="{"EXAMPLE_KEY": "example key"}", but while this throws no error the key does not seem to be available in Jupyter Lab (Jypyternaut works, but the magic commands tell me there's no key).

Is there a way to start a repository via mybinder.org with the same attributes / information (or at the least with the default LLM)?

[krassowski]

You may need to escape the key with single quotes like --AiExtension.default_api_keys='{"EXAMPLE_KEY": "example key"}'

[ssp24]

I've uninstalled my whole Jupyter Lab locally, reinstalled everything, including Jupyter-Ai, and thankfully now --AiExtension.default_language_model was/is working. I noticed there's a config.json now under .local/share/jupyter/jupyter-ai which includes the information on the default LLM as well as the key, so I assume running the command jupyter lab --AiExtension.default_language_model did pass that information to the config.json so the default is now set every time I start jupyter lab, even if I start it without the long option. Is that correct?

If so, is there ANY way to transfer this behaviour to a github-repository/mybinder.org environment? I have tried my luck with a clone of the config.json and an overrides.json and followed instructions on how to insert those config-files with a postBuild or a start file by copying them into various places where mybinder.org/jupyter lab looks for config-files when starting - none of it worked. While jupyter-ai is running perfectly fine on my mybinder.org-repo "from scratch", when I try to use the config-files it mostly leads to the mybinder-repository not being able to load jupyter-ai anymore at all. I assume it might just be a problem with me not having figured out the exact correct location for the file on the mybinder-structure or the config-file not being entirely correct, but I feel like I've tried everything so would really really appreciate it if anyone has any knowledge of or idea where which file might need to go for this to work.

My lalst attempt was using a Dockerfile with the following content, but this didn't work either:

FROM jupyter/base-notebook

# Install any additional packages if needed
RUN pip install jupyterlab

# Set the command to run JupyterLab with the specified option
CMD ["jupyter", "lab", "--AiExtension.default_language_model=openai-chat:gpt-4"]