simonw / llm

Access large language models from the command-line
https://llm.datasette.io
Apache License 2.0
4.81k stars 266 forks source link

Enhancement Proposal: Track Head (for backtracking) #634

Open FergusFettes opened 6 days ago

FergusFettes commented 6 days ago

Sometimes it would be nice to step back a single generation. To fix a typo or redo a generation. This PR makes that possible. This is done by adding a parent ID to the responses and adding a table to track the HEAD ID.

I can clean it up and add some tests, but first I wanted to check if this was something you would be amenable to.

Here is a demonstration of what it allows for:

> llm 'Hello World. Reply with "hello".'
hello
> llm --continue 'Magic word is pineapple. Confirm with "okay".'
okay
> llm --continue 'What is the magic word?'
pineapple
> llm head back
Head moved back to response 01jctw7f1a2xv2n8ckjyhxr3k7
> llm head back
Head moved back to response 01jctw6rh9pd2kvphs0sdcjz8e
> llm --continue 'What is the magic word?'
please
> llm head set 01jctw7f1a2xv2n8ckjyhxr3k7
Head is now at response 01jctw7f1a2xv2n8ckjyhxr3k7
> llm --continue 'What is the magic word?'
pineapple

I think this is useful on its own and not too big of a change. However, this would also make it possible to explore many branches of a conversation in parallel, in a loom-like manner. I would make that as an additional module, but for that to be possible the responses need parent ids, so at least that much would have to be added to the db I believe.

FergusFettes commented 6 days ago

Oh, I also changed the way 'llm --continue' works. Currently it tracks the conversation with the most recent start time. I found it much more helpful to track the conversation that has the most recent generations.