0xacx
commented
1 year ago Hm, that's an interesting edge case.
First, you don't need the prompt flag -p, when you pass the prompt through pipe.
The code hangs because the script part that handles the gpt-3.5-turbo and gpt-4 models, is designed to only work with conversational context, meaning sending back every question and answer to the model, so it can remember the previous messages. As a result it expects the max tokens to be set to at least 100. When you set a limit lower than that, the loop that trims the prompt length hangs. I will have to think the best way to handle this and get back to it. Meanwhile if you need to run something with low token limit, you can try all the other models except gpt-3.5-turbo and gpt-4. The other models do not maintain conversational context by default, only if set (--chat-context or -c flag).
I want the program to exit and not hang