Max tools > 9

[gautierdag]

Hi, Found this library through your post, and I just wanted to say a great job at hitting it on the nail!

Reading the docs (https://github.com/minimaxir/simpleaichat/blob/db19d26250cdee7db8acb52027632083e83aeb28/PROMPTS.md?plain=1#L59C1-L59C185):

- The call sets `{"max_tokens": 1}` so it will only output one number (hence there is a hard limit of 9 tools), which makes it more cost and speed efficient than other implementations.

I couldn't help but point out that the tokenization scheme in gpt-3.5-turbo encodes the first 999 numbers as single tokens. Therefore, up to 999 tools should be possible using "max_tokens":1. It would just mean that you'd need to deal differently with the logit bias since the tokens for n > 9 are indeed not sequential. So the trick below wouldn't work:

https://github.com/minimaxir/simpleaichat/blob/db19d26250cdee7db8acb52027632083e83aeb28/simpleaichat/chatgpt.py#L189)

Feel free to close this.

[minimaxir]

No, this is fair.

The additional constraints are a) logit_bias does have a limit on the number of tokens you can specify (I think 100) and b) appropriate tool accuracy will be bad when you have a ton of tools so it's mostly pretecting the user from themselves. Even OpenAI Plugins doesn't let you use more than 3.

However, with gpt-3.5-turbo-0613 the latter assumption may not be valid, so it may be worth experimenting.

[arisliang]

Just curious, is there a demo code for making it return 0-9? The README Tools example seems automatically calls the tool I think, but not returning 0-9. I imagine there may be cases where we want to manually make the call ourselves.