[Bug]: Error in UI while response is written: Cannot read properties of undefined (reading 'parts')

rotemdan commented 8 months ago

What happened?

When using the Gemini Pro API, an answer containing a code block begins to appear, but then fails and produces this error instead. Trying to regenerate multiple times doesn't seem to help:

Something went wrong. Here's the specific error message we encountered: Cannot read properties of undefined (reading 'parts')

Screenshot_6

Steps to Reproduce

not 100% sure

Use the Gemini Pro API (?)
Ask Gemini a question
The error may appear when the response is being written. (maybe has to do with a code block?)

What browsers are you seeing the problem on?

Brave

Relevant log output

connection is opened
index-RujDGMcq.js:2 error in opening conn.

{
    "sender": "Gemini",
    "messageId": "6da75607-cb1e-4114-bb92-3871191638ea",
    "conversationId": "b1813aba-ebe4-4139-8d48-3d0b5fbd1186",
    "parentMessageId": "e8048ee6-4c28-4709-8387-f50072b55815",
    "unfinished": false,
    "cancelled": false,
    "error": true,
    "final": true,
    "text": "Cannot read properties of undefined (reading 'parts')",
    "isCreatedByUser": false
}

Screenshots

No response

Code of Conduct

[X] I agree to follow this project's Code of Conduct

danny-avila commented 8 months ago

It may be an issue with the dependency; there were some bug fixes they had and I still need to bump the dependency.

For now, are you able to access the error logs of the server? They are located in /api/logs.

if you are using docker, you need to enter the LibreChat container shell

danny-avila commented 8 months ago

Here's a summarized step-by-step guide for inspecting a log file located inside a Docker container when you don't initially know the container's name or ID:

Identify the Container:
- Intention: To find the container where the log file is located.
- Command: Use docker ps to list all running containers and identify the correct one by its name or ID.
```
docker ps
```
Access the Container's Shell:
- Intention: To access the environment inside the container where the log file is located.
- Command: Use docker exec to start an interactive shell session in the container. Replace d3c5726b0144 with the actual container ID.
```
docker exec -it d3c5726b0144 /bin/sh
```
Navigate to the Log File:
- Intention: To locate the log file inside the container.
- Command: Change directory to where the log file is stored, such as /app/api/logs.
```
cd /app/api/logs
```

Use ls to list all log files in the directory

Inspect the Log File:
- Intention: To view or monitor the contents of the log file.
- Commands: Use tools like cat, less, tail, or head to inspect the log file.
  - To view the entire file:
```
cat filename.log
```
  - To view the last few lines:
```
tail -n 100 filename.log
```
  - To open a scrollable interface:
```
less filename.log
```
Remember to replace filename.log with the actual name of the log file.
Exit the Container Shell:
- Intention: To leave the container's environment after inspection.
- Command: Simply type exit.

This process allows you to effectively find and inspect a specific log file inside a Docker container on a headless server.

rotemdan commented 8 months ago

Yes, I'm using Docker Desktop. Here is a screenshot of the error log:

Screenshot_7

Text:

2023-12-28 19:42:09 > LibreChat@0.6.5 backend
2023-12-28 19:42:09 > cross-env NODE_ENV=production node api/server/index.js
2023-12-28 19:42:09 
2023-12-28 19:42:10 2023-12-28 17:42:10 info: `REDIS_URI` not provided, or `USE_REDIS` not set. Redis module will not be initialized.
2023-12-28 19:42:10 2023-12-28 17:42:10 info: Connected to MongoDB
2023-12-28 19:42:11 2023-12-28 17:42:11 info: Server listening on all interfaces at port 3080. Use http://localhost:3080 to access it
2023-12-28 19:48:28 2023-12-28 17:48:28 error: [MeiliMongooseModel.findOneAndUpdate] Convo not found in MeiliSearch and will index 6b921c16-e19a-4d5b-aeed-1f8d1acc1166 Document `6b921c16-e19a-4d5b-aeed-1f8d1acc1166` not found.
2023-12-28 19:51:45 2023-12-28 17:51:45 error: [MeiliMongooseModel.findOneAndUpdate] Convo not found in MeiliSearch and will index e740a1a0-114a-4732-9655-5b31f15c2ea1 Document `e740a1a0-114a-4732-9655-5b31f15c2ea1` not found.
2023-12-28 19:53:21 2023-12-28 17:53:21 error: [MeiliMongooseModel.findOneAndUpdate] Convo not found in MeiliSearch and will index ae6a1194-b386-4512-94d0-e7784821a516 Document `ae6a1194-b386-4512-94d0-e7784821a516` not found.
2023-12-28 19:55:22 2023-12-28 17:55:22 error: [MeiliMongooseModel.findOneAndUpdate] Convo not found in MeiliSearch and will index 979cbb4b-e706-4d28-84be-ed7c88332abc Document `979cbb4b-e706-4d28-84be-ed7c88332abc` not found.
2023-12-28 19:57:25 2023-12-28 17:57:25 error: [MeiliMongooseModel.findOneAndUpdate] Convo not found in MeiliSearch and will index 26d64ae3-d740-4520-ac49-b603c00be7d3 Document `26d64ae3-d740-4520-ac49-b603c00be7d3` not found.
2023-12-28 19:59:06 2023-12-28 17:59:06 error: [MeiliMongooseModel.findOneAndUpdate] Convo not found in MeiliSearch and will index 6b9968dc-1d69-4083-bff9-6936609719b9 Document `6b9968dc-1d69-4083-bff9-6936609719b9` not found.
2023-12-28 20:03:41 2023-12-28 18:03:41 error: [MeiliMongooseModel.findOneAndUpdate] Convo not found in MeiliSearch and will index 9e4822ff-c4f5-4d7b-a54e-f17e3e5c61a3 Document `9e4822ff-c4f5-4d7b-a54e-f17e3e5c61a3` not found.
2023-12-28 21:03:46 2023-12-28 19:03:46 error: [MeiliMongooseModel.findOneAndUpdate] Convo not found in MeiliSearch and will index b1813aba-ebe4-4139-8d48-3d0b5fbd1186 Document `b1813aba-ebe4-4139-8d48-3d0b5fbd1186` not found.
2023-12-28 21:03:54 2023-12-28 19:03:54 error: [handleAbortError] response error and aborting request Cannot read properties of undefined (reading 'parts')
2023-12-28 21:03:54 2023-12-28 19:03:54 error: [handleAbortError] error while trying to abort message Cannot destructure property 'abortController' of 'abortControllers.get(...)' as it is undefined.
2023-12-28 21:05:57 2023-12-28 19:05:57 error: [handleAbortError] response error and aborting request Cannot read properties of undefined (reading 'parts')
2023-12-28 21:05:57 2023-12-28 19:05:57 error: [handleAbortError] error while trying to abort message Cannot destructure property 'abortController' of 'abortControllers.get(...)' as it is undefined.
2023-12-28 21:06:08 2023-12-28 19:06:08 error: [handleAbortError] response error and aborting request Cannot read properties of undefined (reading 'parts')
2023-12-28 21:06:08 2023-12-28 19:06:08 error: [handleAbortError] error while trying to abort message Cannot destructure property 'abortController' of 'abortControllers.get(...)' as it is undefined.
2023-12-28 21:12:47 2023-12-28 19:12:47 error: [handleAbortError] response error and aborting request Cannot read properties of undefined (reading 'parts')
2023-12-28 21:12:47 2023-12-28 19:12:47 error: [handleAbortError] error while trying to abort message Cannot destructure property 'abortController' of 'abortControllers.get(...)' as it is undefined.
2023-12-28 21:18:28 2023-12-28 19:18:28 error: [handleAbortError] response error and aborting request Cannot read properties of undefined (reading 'parts')
2023-12-28 21:18:28 2023-12-28 19:18:28 error: [handleAbortError] error while trying to abort message Cannot destructure property 'abortController' of 'abortControllers.get(...)' as it is undefined.

Very fast response! even before I was able to press "send"!

danny-avila commented 8 months ago

Yeah looks like it could be a dependency error or Google error. Does this happen with every message you send? Need a bit more guidance on reproducing

rotemdan commented 8 months ago

I started using the application about an hour ago, to test the Gemini Pro API. I made some conversations with no problems. The problem started when the response contained code.

Here is the full conversation with Gemini, that produced the error (a very simple test):

[USER] Here is Python code that performs binary search. Can you find errors in it?

def binary_search(arr, low, high, x):
   if high >= low:
       mid = (high + low) // 2
       if arr[mid] == x:
           return mid
       elif arr[mid] > x:
           return binary_search(arr, low, mid - 1, x)
       else:
           return binary_search(arr, mid + 1, high, x)
   else:
       return -1

[Gemini] Yes, there is an error in one of the conditionals.

[User] Can you show the exact error?

[Gemini] Something went wrong. Here's the specific error message we encountered: Cannot read properties of undefined (reading 'parts')

danny-avila commented 8 months ago

I started using the application about an hour ago, to test the Gemini Pro API. I made some conversations with no problems. The problem started when the response contained code.

Here is the full conversation with Gemini, that produced the error (a very simple test):

[USER] Here is Python code that performs binary search. Can you find errors in it?
def binary_search(arr, low, high, x):
   if high >= low:
       mid = (high + low) // 2
       if arr[mid] == x:
           return mid
       elif arr[mid] > x:
           return binary_search(arr, low, mid - 1, x)
       else:
           return binary_search(arr, mid + 1, high, x)
   else:
       return -1
[Gemini] Yes, there is an error in one of the conditionals.

[User] Can you show the exact error?

[Gemini] Something went wrong. Here's the specific error message we encountered: Cannot read properties of undefined (reading 'parts')

Thanks! Should be easy to debug then when I get the chance

danny-avila commented 8 months ago

After trying to debug this, it seems your specific prompt about binary search triggers a Google GenerativeAI recitation error. It's a filter to stop the AI from spewing out copyrighted material, so you need to get it to generate something slightly different.

I'm going to see if I can improve the handling here but ultimately a change is needed in the langchainjs library.

rotemdan commented 8 months ago

Thanks for trying to debug this.

I am getting the same problem with a much simpler prompt:

Basically asking "Can you show a quicksort implementation in python?" (no other context and no system prompt)

Screenshot_8

I also tried with different temperatures, including 1.0, to increase randomness and maybe avoid what you are describing, but it doesn't seem to make a difference.

So it's possible that this problem occurs frequently with code. I am not seeing the same issue via the Bard interface, though.

Google AI studio, when prompting Gemini Pro, does return the result, but truncates it, even if I increase the output length to 16000 (I tried more than 3 times to regenerate it):

Screenshot_10

The truncation doesn't occur with other coding prompts I've tried, for heapsort, merge sort, bubble sort, binary search, etc.

So it's possible that the problem occurs with Gemini Pro in general, but not in Bard.

danny-avila commented 8 months ago

Google AI studio, when prompting Gemini Pro, does return the result, but truncates it, even if I increase the output length to 16000 (I tried more than 3 times to regenerate it): ... So it's possible that the problem occurs with Gemini Pro in general, but not in Bard.

Yeah, truncating the response is something I'm working on right now, which would be appreciated app-wide for all errors mid-generation.

It's interesting that its triggering a "copyrighted material" filter, and only via API, probably trying to hide the fact that it was trained on such? More likely, it is trying to protect users of its API (enterprises running their own products) from using any copyright material that only Google has the right to.

But with code, and data structures and algorithms specifically, this makes sense. There is usually a "best" way to slice it the most time/space efficient manner with not much variation to the code, so it's an easy way to trigger the censor.

danny-avila / LibreChat