Idea: pipe data to `llm chat` to start the conversation

[simonw]

Might be useful to be able to do this:

git diff | llm chat -m gpt-4 --in

Where the content doesn't actually get sent to the model at all (yet), but is instead added to the history and made ready to be sent with the first question that the user asks.

[simonw]

Since llm chat itself starts an interactive session that reads from standard input, I'm not sure how feasible this actually is. There might be a clever trick involving if not sys.stdin.isatty() that can do it though.

[vividfog]

Very useful as it stands. 💡

A later evolution could be a form of llm embed which chops the input and now you are chatting with top k chunks of that file. Then it can be far longer than the context window.

cat massive.log | llm chat --with-a-big-file

Lacking more options, it would use default chunk size, overlap, algo etc. Which you would inherit from a future embed --chunk feature, once that exists first.

This is effectively a form of RAG.

A runtime command like

!fetch now explain any Bluetooth errors in the log

would fetch new chunks on demand.

[simonw]

That's really interesting. I've been thinking about RAG more in terms of starting with you creating a collection (with llm embed and llm embed-multi) and then having a conversation against that (or multiple) collections, but being able to start a chat directly against one or more file - like all those popular "chat with this PDF" services - could be a neat feature too.

[ebetica]

It would be great to have this capability - I think an extra command line argument would work, something like --init-prompt-from-file filename.txt. You can then have something like llm chat --init-prompt-from-file massivelog or llm chat -i massive.log. Would you accept the PR if I implemented it?