getumbrel / llama-gpt

A self-hosted, offline, ChatGPT-like chatbot. Powered by Llama 2. 100% private, with no data leaving your device. New: Code Llama support!
https://apps.umbrel.com/app/llama-gpt
MIT License
10.73k stars 696 forks source link

/v1/chat/completions max_token should accept null #113

Open Ale4224 opened 12 months ago

Ale4224 commented 12 months ago

Calling the api with max_token null should work, but i get a 500 http response

per openai api doc https://platform.openai.com/docs/api-reference/chat/create#chat/create-max_tokens it should accept the null and fallback to default infinity

I could understand the default to 16 instead of infinity, but it should not break

{
    "error": {
        "message": "[{'type': 'int_type', 'loc': ('body', 'max_tokens'), 'msg': 'Input should be a valid integer', 'input': None, 'url': 'https://errors.pydantic.dev/2.4/v/int_type'}]",
        "type": "internal_server_error",
        "param": null,
        "code": null
    }
}

not sure if this is the api response or the error message that the client I'm using is logging, but the error is clear If i set the max_token with a valid integer i get no errors