API examples in docs outdated

[Technologicat]

Describe the bug

The Python examples in the wiki docs are out of date, and crash with various errors.

For example:

• Python chat example: crashes mysteriously, actually due to a 500 error from the server.
• Python chat example with streaming: raises KeyError, payload does not have key "message".

Note that my text-generation-webui installation itself works fine, including the API - SillyTavern can use it just fine.

I'd like to use text-generation-webui as an LLM backend in a small custom project. Right now, because the examples are not working, it is difficult to build a custom program that calls the API, short of reading through the (much longer) source code of SillyTavern or another known-working app.

Is there an existing issue for this?

• [X] I have searched the existing issues

Reproduction

1. Start text-generation-webui with the openai extension enabled, to serve the API on localhost:5000.
2. Try to run the Python code examples from the wiki docs.

Screenshot

No response

Logs

Python chat example

> hello
Traceback (most recent call last):
  File "/home/xxx/.local/lib/python3.10/site-packages/requests/models.py", line 971, in json
    return complexjson.loads(self.text, **kwargs)
  File "/usr/lib/python3/dist-packages/simplejson/__init__.py", line 525, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3/dist-packages/simplejson/decoder.py", line 370, in decode
    obj, end = self.raw_decode(s)
  File "/usr/lib/python3/dist-packages/simplejson/decoder.py", line 400, in raw_decode
    return self.scan_once(s, idx=_w(s, idx).end())
simplejson.errors.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/xxx/.../example.py", line 25, in <module>
    assistant_message = response.json()['choices'][0]['message']['content']
  File "/home/xxx/.local/lib/python3.10/site-packages/requests/models.py", line 975, in json
    raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

This is actually caused by a 500 error from the server, which is not detected, since the example has no error handling (hence the misleading error messages above).

By adding print(response) right after the line response = requests.post(url, headers=headers, json=data, verify=False), the program prints <Response [500]> before crashing as above.

Python chat example with streaming

> hello
Traceback (most recent call last):
  File "/home/xxx/.../example.py", line 61, in <module>
    chunk = payload['choices'][0]['message']['content']
KeyError: 'message'

System Info

OS: Linux Mint 21.1
GPU brand: NVIDIA
GPU model: Geforce RTX 3070 Ti Laptop GPU (8 GB)
text-generation-webui version: d011040f (v1.13)

EDIT: Fixed broken formatting, due to the auto-formatting in the logs and system info sections.

[Technologicat]

Specifically related to Python chat example:

Someone posted in April 2024 on Reddit: OpenAI API chat with character examples no longer works. However, that fix has already been pushed.

But I figured it out - the problem was that the chosen AI character must exist. Changing "character": "Example" to "character": "Assistant", the Python chat example works. I think this should be mentioned in the docs.

EDIT: And actually, the server did print this in its console:

12:39:33-097268 ERROR    Could not find the character "Example" inside characters/. No character has been loaded. 

I suppose I should keep that window visible on a separate screen when testing things like this. :)

[Technologicat]

And specifically related to Python chat example with streaming:

It seems the "message" field has been renamed to "delta". Makes sense given how it's used.

So to fix this example, change the line

        chunk = payload['choices'][0]['message']['content']

to

        chunk = payload['choices'][0]['delta']['content']