getting "all messages must have non-empty content" with `claude-3-5-sonnet-latest`

[gordonwoodhull]

Hi Carson!

Thanks to chatlas, I have my quarto chatbot working on both anthropic and openai! ✨

https://github.com/gordonwoodhull/quarto-data-science-chatbot/

It took less than an hour to port from the openapi python library, and the output tool show_answer is connected much more reliably.

Chatlas also multiplexes the streaming chat and sending the documents via the tool very nicely!

I find that with anthropic claude-3-opus-20240229 my chatbot works reliably but it is very chatty. If I switch to claude-3-5-sonnet-latest with

QUARTO_DS_CHATBOT_PROVIDER=anthropic QUARTO_DS_CHATBOT_MODEL=claude-3-5-sonnet-latest shiny run ds-quarto-chatbot.py

It seems both speedier and more to-the-point, but when I ask it to revise its answer, my second input, it crashes with

Error code: 400 - {'type': 'error', 'error': {'type': 'invalid_request_error',
   'message': 'messages.3: all messages must have non-empty content except for the optional final assistant message'}}

Typical conversation:

1. please draw a heatmap of synthetic randomized hourly temperatures for a place that's in the 70s in the afternoon and gets down to 40 at night, using ggplot2 (or altair, or plotnine, etc.)

2. yeah that's great but please put the days in x and the hours in y and nights should be colder than days (or whatever silly thing it got wrong)

My chatbot crashes consistently on step 2 with sonnet, but works reliably with opus and with openai gpt-4o. I haven't tested others yet.

Here is the full stack trace:

Traceback (most recent call last):
  File "/Users/gordon/src/quarto-cli/.venv/lib/python3.12/site-packages/shiny/reactive/_extended_task.py", line 131, in _execution_wrapper
    return await self._func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/gordon/src/quarto-cli/.venv/lib/python3.12/site-packages/shiny/ui/_chat.py", line 568, in _stream_task
    await self._append_message_stream(message)
  File "/Users/gordon/src/quarto-cli/.venv/lib/python3.12/site-packages/shiny/ui/_chat.py", line 595, in _append_message_stream
    async for msg in message:
  File "/Users/gordon/src/quarto-cli/.venv/lib/python3.12/site-packages/shiny/_utils.py", line 508, in __anext__
    value = next(self.iterator)
            ^^^^^^^^^^^^^^^^^^^
  File "/Users/gordon/src/quarto-cli/.venv/lib/python3.12/site-packages/chatlas/_chat.py", line 902, in __next__
    chunk = next(self._generator)
            ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/gordon/src/quarto-cli/.venv/lib/python3.12/site-packages/chatlas/_chat.py", line 632, in _chat_impl
    for chunk in self._submit_turns(
                 ^^^^^^^^^^^^^^^^^^^
  File "/Users/gordon/src/quarto-cli/.venv/lib/python3.12/site-packages/chatlas/_chat.py", line 681, in _submit_turns
    response = self.provider.chat_perform(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/gordon/src/quarto-cli/.venv/lib/python3.12/site-packages/chatlas/_anthropic.py", line 244, in chat_perform
    return self._client.messages.create(**kwargs)  # type: ignore
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/gordon/src/quarto-cli/.venv/lib/python3.12/site-packages/anthropic/_utils/_utils.py", line 274, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/gordon/src/quarto-cli/.venv/lib/python3.12/site-packages/anthropic/resources/messages.py", line 888, in create
    return self._post(
           ^^^^^^^^^^^
  File "/Users/gordon/src/quarto-cli/.venv/lib/python3.12/site-packages/anthropic/_base_client.py", line 1277, in post
    return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/gordon/src/quarto-cli/.venv/lib/python3.12/site-packages/anthropic/_base_client.py", line 954, in request
    return self._request(
           ^^^^^^^^^^^^^^
  File "/Users/gordon/src/quarto-cli/.venv/lib/python3.12/site-packages/anthropic/_base_client.py", line 1058, in _request
    raise self._make_status_error_from_response(err.response) from None
anthropic.BadRequestError: Error code: 400 - {'type': 'error', 'error': {'type': 'invalid_request_error', 'message': 'messages.3: all messages must have non-empty content except for the optional final assistant message'}}

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/gordon/src/quarto-cli/.venv/lib/python3.12/site-packages/shiny/reactive/_reactives.py", line 584, in _run
    await self._fn()
  File "/Users/gordon/src/quarto-cli/.venv/lib/python3.12/site-packages/shiny/ui/_chat.py", line 583, in _handle_error
    await self._raise_exception(e)
  File "/Users/gordon/src/quarto-cli/.venv/lib/python3.12/site-packages/shiny/ui/_chat.py", line 345, in _raise_exception
    raise NotifyException(str(e), sanitize=sanitize) from e
shiny.types.NotifyException: Error code: 400 - {'type': 'error', 'error': {'type': 'invalid_request_error', 'message': 'messages.3: all messages must have non-empty content except for the optional final assistant message'}}

[gordonwoodhull]

Aha, it also works reliably with claude-3-5-sonnet-20240620, only failing with latest = claude-3-5-sonnet-20241022

[cpsievert]

Very nice, thanks for sharing!

I'll eventually have a look into, but in the meantime you can utilize the new .chat(echo="all") argument to get a better sense of what the conversation looks like before the error occurs, which should help highlight where/why things are going wrong, and thus help to get a more minimal reproducible example

[gordonwoodhull]

Great! I'll try to use echo="all" to provide a reproducible example.

In the meantime, I hope the hint to try earlier sonnet is helpful to anyone who hits this error. Older sonnet is good enough for my development purposes, no complaints here.

Thanks so much for chatlas!