Open thiswillbeyourgithub opened 11 months ago
Update : I did try to make something and run into major issues.
Basically when you use a pipe, click fails. It's been unfixed since at least 2020.
In my code below as soon as python gets to click.prompt
in line 441 I get a Aborted!
error if the pipe is used.
The reasons of this very silent error is because of click's exception handling. But I couldn't get a way to use the standalone_mode
to get a more explicit exception. I'm not familiar with click and don't know where exactly is this command.main
in all that...
In the end I gave up also because when replacing click.prompt()
by input()
I got Aborted!
too and there's no way to open a breakpoint()
in this kind of click context apparently. Very frustrating.
Not using a pipe and just relying on a first_user_message
option would be an easier workaround.
Here's my patch of code anyway:
diff --git a/llm/cli.py b/llm/cli.py
index 3fa2ecc..55f0c52 100644
--- a/llm/cli.py
+++ b/llm/cli.py
@@ -294,6 +294,7 @@ def prompt(
@cli.command()
@click.option("-s", "--system", help="System prompt to use")
+@click.argument("use_stdin", required=False)
@click.option("model_id", "-m", "--model", help="Model to use")
@click.option(
"_continue",
@@ -329,6 +330,7 @@ def prompt(
@click.option("--key", help="API key to use")
def chat(
system,
+ use_stdin,
model_id,
_continue,
conversation_id,
@@ -405,6 +407,29 @@ def chat(
if not should_stream:
validated_options["stream"] = False
+ if use_stdin:
+ def read_stdin():
+ nonlocal prompt
+
+ # Is there extra prompt available on stdin?
+ stdin_prompt = None
+ if not sys.stdin.isatty():
+ stdin_prompt = sys.stdin.read()
+
+ if stdin_prompt:
+ bits = [stdin_prompt]
+ prompt = " ".join(bits)
+
+ if prompt is None and sys.stdin.isatty():
+ # Hang waiting for input to stdin (unless --save)
+ prompt = sys.stdin.read()
+ return prompt
+ first_message = read_stdin().strip()
+ if not first_message:
+ click.echo("Nothing found in stdin.")
+ else:
+ first_message = None
+
click.echo("Chatting with {}".format(model.model_id))
click.echo("Type 'exit' or 'quit' to exit")
click.echo("Type '!multi' to enter multiple lines, then '!end' to finish")
@@ -412,7 +437,13 @@ def chat(
accumulated = []
end_token = "!end"
while True:
- prompt = click.prompt("", prompt_suffix="> " if not in_multi else "")
+ if use_stdin:
+ prompt = click.prompt("", prompt_suffix="> " if not in_multi else "")
+ else:
+ click.echo(f"> {first_message}\n")
+ prompt = first_message
+ use_stdin = False
+
if prompt.strip().startswith("!multi"):
in_multi = True
bits = prompt.strip().split()
Hi,
I've been tinkering lately with using a meta prompt to generate better prompts.
The idea is roughly that you give
python for fibonacci sequence
to an llm that takes as system promptYou are a master of the education system. Given a student request, you know how to reformulate it to add details and phrasing so that the highly regarded teacher receiving the request will answer is perfectly.
and as promptpython for fibonacci sequence
. The result isCould you recommend some resources or provide guidance on learning how to use Python to implement the Fibonacci sequence algorithm?
That can then be sent to the actual chat.(the example is terrible but I think that with a good system prompt it can actually be awesome)
Obviously being a huge fan of
llm
my first idea was to use templates or pipes and voila.Unfortunately, I can specify the system and prompt of the "master of the education system" but when I want to give it to the actual chat I can't because I don't think you can currently specify the first message of a chat message. So I end up specifying it to the system prompt and just typing "go" manually as the first message but it's quite inelegant.
So I think it would be incredible to be able to specify in the cli some prompt messages instead of just the system prompt. What do you think :)?