Add max tokens to ModelType

[Blarc]

Would it be possible to add max tokens property to ModelType enum, that specifies the maximum number of tokens that the model can consume?

With this property added, one could add a method for checking if prompt is too big for a given model.

I've extracted max tokens for models from OpenAI's page:

gpt-4;8,192
gpt-4-0314;8,192
gpt-4-32k;32,768
gpt-4-32k-0314;32,768
gpt-3.5-turbo;4,096
gpt-3.5-turbo-0301;4,096
text-davinci-003;4,097
text-davinci-002;4,097
code-davinci-002;8,001
text-curie-001;2,049
text-babbage-001;2,049
text-ada-001;2,049
davinci;2,049
curie;2,049
babbage;2,049
ada;2,049
code-davinci-002;8,001
code-davinci-001;8,001
code-cushman-002;2,048
code-cushman-001;2,048

I am not sure what are the limitations for other models though...

[tox-p]

Sounds reasonable 👍 I would not want to add the temporary snapshot versions of GPT3 and 4 to the enum to prevent having to update the library every three months when they deprecate. But for the rest of the models, we can totally add the max tokens. The only ones that seem to miss a documented token limit are davinci-codex, cushman-codex, text-davinci-001, text-davinci-edit-001 and code-davinci-edit-001.

I will look into those tomorrow

[tox-p]

The change is included in the 0.3.0 release

It should soon be available on maven central

Thank you, for your suggestion 🙂