Closed simonw closed 1 year ago
I need a top-level design that supports the following:
llm bard
and llm claude
and suchlike - some of these may be added using plugins.llm "prompt goes here"
default command can be configured to use different models and model settingsMaybe llm chatgpt
becomes llm openai
.
I need to figure out how the completion APIs like GPT-3 will work within that llm openai
command though.
What do you think of keeping llm chatgpt
while also adding llm openai
?
llm chatgpt
calls by default gpt-3.5-turbo
llm chatgpt -4
calls gpt-4
llm openai --davinci-003
llm openai --gpt-3.5-turbo
llm openai --gpt-4
Or a longer version (but maybe better for clarity):
llm openai --model gpt-4 "prompt"
llm openai --model davinci-003 "prompt"
As for the "chat mode", it could be done by using the flag -i
or --interactive
.
Also, llm
would use llm chatgpt
by default, but this could be changed in a configuration file.
For ChatGPT, there's also the context length to consider:
llm chatgpt
would still call the "basic" gpt-3.5-turbo
modelllm chatgpt -16k
llm chatgpt -4 -32k
llm openai --gpt-3.5-turbo-16k
llm openai --gpt-4-32k-0613
I'm going to create:a new default command called prompt
- which can take the user's preferences into account before passing on to some other command (just openai
for the moment).
UPDATE: No, that doesn't work - because I need all of the various options and arguments on the command to be available on that default command too. So I should stick with openai
as the default.
Changed my mind again - I think llm prompt
is indeed the way to go here.
I'm going to make that the default command, and have it expose a subset of functionality that I expect to be common across all models - it will accept a prompt and a model and run that prompt.
If you need to do something specialized with custom options, you can use llm name-of-model
instead. llm prompt
will be the lowest common denominator.
I want to make streaming mode the default - I'm fed up of forgetting to add -s
to everything. I don't see any harm in it as a default, people can turn it off with --no-stream
if they really want to.
Here's the set of options for the prompt
command now:
https://github.com/simonw/llm/blob/68c3848eb38e9b32c1273c4cfcb5016d6b5e8d93/llm/cli.py#L32-L51
(I just removed the -4
option).
Do these make sense as a set of options for any generic model?
Looking at them in turn:
prompt
- yes, all LLMs should take a prompt--system
- only relevant to ChatGPT and GPT-4 at the moment, but I think it's OK to leave it in here-m
/--model
- which model to use, fundamental--no-stream
- if we stream by default this makes sense, plus it's OK if it is a no-op-n
/--no-log
- yes, all models should log by default and have an opt-out-c
/--continue
- only makes sense for "chat" oriented models, can throw an error otherwise--chat
- ditto--key
- yes for models which use an API key, ignored for othersI'm happy with these as the standard set of options. I don't think it's too harmful that some of them won't make sense for every model and should return errors if used incorrectly.
As part of the templates feature I'll be adding a -t/--template name-of-template
option, which will definitely work for all models.
Further work on this will happen here:
Originally posted by @simonw in https://github.com/simonw/llm/issues/6#issuecomment-1590638188