nlkitai / nlux

The 𝗣𝗼𝘄𝗲𝗿𝗳𝘂𝗹 Conversational AI JavaScript Library 💬 — UI for any LLM, supporting LangChain / HuggingFace / Vercel AI, and more 🧡 React, Next.js, and plain JavaScript ⭐️
https://docs.nlkit.com/nlux
Other
937 stars 48 forks source link

Intermittent Ignoring of First or Last Chunk in StreamAdapter #86

Closed aspirantzhang closed 2 weeks ago

aspirantzhang commented 2 weeks ago

I am encountering some issues and am quite puzzled. I'm not sure if I'm on the right track, so I need your help to review it.

I wrote a very simple streamAdapter and simulated the 26 English letters, adding them using observer.next. However, I noticed some random problems. Sometimes, the first chunk is ignored, and other times, the last chunk is ignored, as shown in the attached image.

微信截图_20240614134922

I wonder what might be causing this. Could it be that the data is being pushed too quickly?

To address this, I added some delays each time, which seemed to help a bit. After consulting with an AI, I found a solution by using the following method:

observer.next('\u200B');

I inserted this invisible character before and after each processing step, and it seems to work wonders.

However, I am still confused as to why this is happening. 😂

I believe this might be an issue, possibly a significant one, but my understanding is limited. I truly don't comprehend why this is occurring. It seems I need everyone's help.

Reproduction

https://codesandbox.io/p/sandbox/custom-renderer-status-forked-5y2gfz

salmenus commented 2 weeks ago

Thanks for providing a reproducible example @aspirantzhang I'm investigating

salmenus commented 2 weeks ago

I was able to consistently reproduce the issue when I throttle my CPU 6x down.

Screenshot 2024-06-14 at 18 26 04

Screenshot 2024-06-14 at 18 24 59

salmenus commented 2 weeks ago

I just pushed a fixed to version 2.6.6 https://www.npmjs.com/package/@nlux/react

Give it a try and let me know if that fixed your problem @aspirantzhang

aspirantzhang commented 2 weeks ago

I believe the issue has been resolved. Thank you very much.

jrgrez commented 3 hours ago

I'm experiencing the same issue with the javascript library