Closed Frajder closed 1 year ago
tluyben
commented
1 year ago In https://github.com/cogentapps/chat-with-gpt/blob/1be33f5d0b0cdd8b35e7cab1a5d6bdffb6cec2ba/app/src/tokenizer.ts#L49there is code that should limit the messages, but maybe there is a bug or something? Trying to reproduce this.
Frajder
commented
1 year ago @tluyben Thank you for devoting your time to this issue. To reproduce this you just have to have a long enough conversation. Let's see how long mine was.
┌──(frajder㉿workstation)-[/tmp/msg_count]
└─$ user_msg_count=$(cat long_conversation.htm | grep '<div class="message by-user' | wc -l) && \
assistant_msg_count=$(cat long_conversation.htm | grep '<div class="message by-assistant' | wc -l) && \
total_characters=$(python3 -c "import re; print(len(''.join(re.findall(r'<p>(.*?)<\/p>', open('long_conversation.htm', 'r').read(), re.DOTALL))))" | awk '{print $1}') && \
echo "Total number of user messages: $user_msg_count | Total number of assistant messages: $assistant_msg_count | Total characters: $total_characters"
Total number of user messages: 37 | Total number of assistant messages: 37 | Total characters: 28065
┌──(frajder㉿workstation)-[/tmp/msg_count]
└─$ # ok maybe that was not the most reliable way...
┌──(frajder㉿workstation)-[/tmp/msg_count]
└─$ scp w:~/dockers/chatgpt-ui/data/* .
chat.sqlite
object-store.sqlite
┌──(frajder㉿workstation)-[/tmp/msg_count]
└─$ user_msg_count=$(sqlite3 chat.sqlite "select count(id) from messages where chat_id = '8bb13ebe-8e68-400d-b920-bf1002f6cd04' and data like '%\"role\":\"user\"%';") && \
assistant_msg_count=$(sqlite3 chat.sqlite "select count(id) from messages where chat_id = '8bb13ebe-8e68-400d-b920-bf1002f6cd04' and data like '%\"role\":\"assistant\"%';") && \
total_characters=$(sqlite3 chat.sqlite "SELECT json_extract(data, '$.content') as content FROM messages WHERE chat_id = '8bb13ebe-8e68-400d-b920-bf1002f6cd04';" | awk '{ sum += length } END { print sum }')
┌──(frajder㉿workstation)-[/tmp/msg_count]
└─$ echo "Total number of user messages: $user_msg_count | Total number of assistant messages: $assistant_msg_count | Total characters: $total_characters"
Total number of user messages: 37 | Total number of assistant messages: 37 | Total characters: 25259
So, in my case, there were a total of 37 messages sent and received, for a combined total of 74. The total number of characters was somewhere between 25,000 and 30,000.
They said GPT-4 is good for coding. Let's give it a shot, see here: Problem-with-long-conversations_63-GPT4_attempt.md 😂😂😂
cogentapps
commented
1 year ago Thanks for the detailed report!
This should be fixed by https://github.com/cogentapps/chat-with-gpt/commit/b6881c69b4f979e70d8069c70998fd01d5e6ceb9.
Summary
In case of long conversation, the chat suddenly stop working. There is no response or error message, and the spinner keeps showing up in the message field and ChatGPT response.
How to Reproduce
The problem always occurs if we use the same conversation for a long time with a larger number of requests. Or when we send too many characters in one request
Expected Behavior
Continuation of the conversation and correct request shortening. Show an error without sending a request with too many characters.
Actual Behavior
The conversation cannot be continued and it is no longer possible to use this conversation. When the "refresh" button is clicked, an error message that says "Too many requests, please try again later" may appear. It seems like the conversation is not being trimmed properly and the request may have fallen into a loop. It seems that there is also no implemented character limit that can still be sent in one request, such as a request for a summary of too long text. It seems the code for determining how many tokens are required for the conversation according to the number of characters of the request we want to send is not working correctly.
Docker environment
Run docker based on the last Dockerfile.
Additional messages from browser console