Open simonw opened 1 year ago
Since llm chat
itself starts an interactive session that reads from standard input, I'm not sure how feasible this actually is. There might be a clever trick involving if not sys.stdin.isatty()
that can do it though.
Very useful as it stands. 💡
A later evolution could be a form of llm embed which chops the input and now you are chatting with top k chunks of that file. Then it can be far longer than the context window.
cat massive.log | llm chat --with-a-big-file
Lacking more options, it would use default chunk size, overlap, algo etc. Which you would inherit from a future embed --chunk feature, once that exists first.
This is effectively a form of RAG.
A runtime command like
!fetch now explain any Bluetooth errors in the log
would fetch new chunks on demand.
That's really interesting. I've been thinking about RAG more in terms of starting with you creating a collection (with llm embed
and llm embed-multi
) and then having a conversation against that (or multiple) collections, but being able to start a chat directly against one or more file - like all those popular "chat with this PDF" services - could be a neat feature too.
It would be great to have this capability - I think an extra command line argument would work, something like --init-prompt-from-file filename.txt
. You can then have something like llm chat --init-prompt-from-file massivelog
or llm chat -i massive.log
. Would you accept the PR if I implemented it?
Might be useful to be able to do this:
Where the content doesn't actually get sent to the model at all (yet), but is instead added to the history and made ready to be sent with the first question that the user asks.