search

sagemathinc / cocalc

CoCalc: Collaborative Calculation in the Cloud
https://CoCalc.com
Other
1.17k stars 216 forks source link

do not include context in the actual chat log #7892

Open williamstein opened 3 weeks ago

williamstein commented 3 weeks ago

When you use AI features with cocalc the communication is in the side chat, and there's a hidden "details" html element that often has context, e.g., a few hundreds (or thousand lines of code). If you work with this for a few months (say), it really adds up! Also, odds are nobody ever looks at it.

We should have two versions of a message when working with AI:

The one that goes in the log doesn't need all that context; it's a waste of space and very rarely matters. The same data is in the notebook (via timetravel). What would be better might be to just store the cell id's, which is technically the same info...

@haraldschilly thoughts? This is causing major pain for at least one of our longterm users.

WORKAROUND: Copy your ipynb file to a new file so you get a new side chat.

I'm not labeling this "Jupyter" because it is a general problem for all document types.

williamstein commented 3 weeks ago

I just worked on this for a bit, and realized there is a major detail, which is that you can't "regenerate response" if you don't have the full code. Best idea I have -- we could store code in memory in the browser, but disable "regenerate response" once browser refresh happens (or maybe old code is removed from an LRU cache).

williamstein commented 3 weeks ago

I did some work on this, but it isn't worth the effort right now: https://github.com/sagemathinc/cocalc/pull/7893

There's a lot of things to worry about that are subtle. E.g, sending the whole conversation history is complicated if you have removed some of the log from the past... Also, dealing with the fact that any message can be edited at any time. Putting everything together just makes implementing this really hard without introducing confusing bugs or horrible code. And is it really worth it? Not clear.

williamstein commented 3 weeks ago

This isn't a bug but a performance issue. We don't seem to have a tag for performance?

williamstein commented 2 weeks ago

Related to this -- I tried using AI to draw a plot just now in a notebook, and the code looked OK, but the result sucked... but there is no way to "regenerate" after you actually run the code or know what I requested. I thus had to full type in my question again. The "Generate code cell" functionality is the only place in cocalc where I lose my work and have to type something I've already typed, etc. Something minimal recording that you used AI and your question could be logged to the side chat.

If for the input we always just linked to a specific place and revision of the original notebook, rather than actually copying code into the chat, that would massively decrease the size of the log, but we would not lose any actual information. TimeTravel exists -- let's use it!