cell magics

[jeisner]

You might consider adding support for some cell magics (which are kernel-specific). I tried executing %pwd and !pwd from a ChatLLM cell, but they were just fed to the LLM.

[jeisner]

BTW, the reason I was trying these was that {{%% include sys.prompt %%}} is not working for me in either a ChatLLM System Prompt cell or a ChatLLM cell. So I was trying to figure out why not -- I have sys.prompt in the same directory as the notebook.

[jeisner]

Anyway, cell magics might be a convenient way for a user to temporarily or persistently change the the settings (temperature, model, truncation, etc.).

[zkolter]

Ok, so lengthy discussion here, which is all going to come across like a big excuse, but hear me out :-).

Originally I had thought to implement this all with a cell-magic-like interface, so that these basically just worked within normal Jupyter notebooks. You could then use cell magics, and Python cells, etc, all as in a usual notebook. But, and maybe this was a mistake: 1) it seemed a bit less clean, because I couldn't figure out how to as-easily not override all the features of typical notebooks with my own settings (like the different renderer, the model selection for blocks, etc).

So the upshot here is that .chatllm notebooks are not Jupyter notebooks in the end. They have the same notebook format (which is really VSCode's interface), but there is no backing kernel, no default support for cell magic, that kind of thing. It means it's really a different environment entirely, which in some ways feels a lot cleaner to me, but is also a bit of a confusing UX for those who are going to come to this assume it's a typical notebook.

I'm not entirely sure if going a more pure Jupyter notebook plugin/extension (even just in VSCode) would be the better route. I'll look into whether cell magics can be supported separately even for a normal notebook.

[zkolter]

BTW, the reason I was trying these was that {{%% include sys.prompt %%}} is not working for me in either a ChatLLM System Prompt cell or a ChatLLM cell. So I was trying to figure out why not -- I have sys.prompt in the same directory as the notebook.

Oh, and also, the reason this likely isn't working is that links are all relatively to the open VSCode workspace folder. So you'll need to run the "Open Folder" command in VScode, and ensure that your links are relatively to that. Another minor annoyance of not actually having a backing kernel. Maybe there is an easier way around this, though, it feels like most people are virtually always going to want/expect links relative to the notebook, and not the project workspace (which then would require always opening the project workspace).

[jeisner]

https://ipython.readthedocs.io/en/stable/interactive/magics.html says:

Whether Magics are available on a kernel is a decision that is made by the kernel developer on a per-kernel basis. To work properly, Magics must use a syntax element which is not valid in the underlying language. ... You can create and register your own Magics with IPython.

So I think even if you were using a kernel, you would have to write code to parse out the magic directives yourself. Just as you currently parse include.

I'll look into whether cell magics can be supported separately even for a normal notebook.

I don't know whether you can automatically import other people's magics, but presumably you should be able to implement your own, e.g. %%temperature 0 I mean, what's stopping you from parsing the cell and running whatever code you want?

[zkolter]

Yep, I can definitely parse cell magics if needed (my comment was only that I had thought that certain magics like %%pwd etc used the kernel, but this may not be true). I'm mainly trying to decide if it's a good idea overall...

One thing to note: for your temperature example, you know that you can add this via the model settings, right? If you open the model settings, you can add a temperature parameter. It seems a bit cleaner to me to just have a separate model you'd run with temp=0 rather than have a separate cell magic. Because different models have different parameters/names, and which of these should correspond to cell magics or not?

I'll think about it, though. I see the value in easily specifying some of the more "common" settings, I just worry a bit about feature creep if we start to define a lot of magics (and how to determine which parameters "deserve" their own magic, etc).

BTW, I did fix the relative file location issue in #6 (and the bug in the model names in the example files). This is merged into main, and I'll update the plugin on the marketplace shortly.

[jeisner]

My thinking was that you might want to use different temperatures in different cells (and be able to see in the notebook that you did that).

Not limited to temperature, of course. You might switch among the registered models in the same way. As you say, that might also change the default temperature.

which parameters "deserve" their own magic

That's easy - all of them!

It's possible that this kind of thing is more useful to people who are using the notebook to experiment with prompt engineering for building reusable systems. Which may not be a scenario that you're trying to support. You may be more focused on users who are just trying to talk to ChatGPT to get some ad hoc work done once. Folks who are doing prompt engineering may instead want to use an ordinary Python notebook with a package like guidance -- that gives them more fine grained control, including the ability to store inputs and outputs in variables.

[jeisner]

BTW, I did fix the relative file location issue in https://github.com/locuslab/chatllm-vscode/pull/6 (and the bug in the model names in the example files)

Yes, I saw, thanks!