search

ripperhe / Bob

Bob 是一款 macOS 平台的翻译和 OCR 软件。
https://bobtranslate.com
8.96k stars 511 forks source link

`streamHandler` 返回的 `streamData` 存在截断问题 #542

Closed liby closed 1 year ago

liby commented 1 year ago

运行环境

描述问题

在请求经过代理的 OpenAI API 时，streamData.text 出现截断的问题，模板项目是 chatbot-ui，相关的代码在 chatbot-ui/blob/main/pages/api/chat.ts

image

通过 Console 看到失败的日志 log，log 是在这一行下面打印的

image

通过抓包看到 Response 返回的数据没有问题：

image
Response ``` data: {"id":"chatcmpl-7KJM8RQ9fDHPB7WbhCevAGbbRRiGW","object":"chat.completion.chunk","created":1685075320,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"role":"assistant"},"index":0,"finish_reason":null}]} data: {"id":"chatcmpl-7KJM8RQ9fDHPB7WbhCevAGbbRRiGW","object":"chat.completion.chunk","created":1685075320,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"我们"},"index":0,"finish_reason":null}]} data: {"id":"chatcmpl-7KJM8RQ9fDHPB7WbhCevAGbbRRiGW","object":"chat.completion.chunk","created":1685075320,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"记录"},"index":0,"finish_reason":null}]} data: {"id":"chatcmpl-7KJM8RQ9fDHPB7WbhCevAGbbRRiGW","object":"chat.completion.chunk","created":1685075320,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"您"},"index":0,"finish_reason":null}]} data: {"id":"chatcmpl-7KJM8RQ9fDHPB7WbhCevAGbbRRiGW","object":"chat.completion.chunk","created":1685075320,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"的"},"index":0,"finish_reason":null}]} data: {"id":"chatcmpl-7KJM8RQ9fDHPB7WbhCevAGbbRRiGW","object":"chat.completion.chunk","created":1685075320,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"API"},"index":0,"finish_reason":null}]} data: {"id":"chatcmpl-7KJM8RQ9fDHPB7WbhCevAGbbRRiGW","object":"chat.completion.chunk","created":1685075320,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"使用"},"index":0,"finish_reason":null}]} data: {"id":"chatcmpl-7KJM8RQ9fDHPB7WbhCevAGbbRRiGW","object":"chat.completion.chunk","created":1685075320,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"数据"},"index":0,"finish_reason":null}]} data: {"id":"chatcmpl-7KJM8RQ9fDHPB7WbhCevAGbbRRiGW","object":"chat.completion.chunk","created":1685075320,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"，"},"index":0,"finish_reason":null}]} data: {"id":"chatcmpl-7KJM8RQ9fDHPB7WbhCevAGbbRRiGW","object":"chat.completion.chunk","created":1685075320,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"例如"},"index":0,"finish_reason":null}]} data: {"id":"chatcmpl-7KJM8RQ9fDHPB7WbhCevAGbbRRiGW","object":"chat.completion.chunk","created":1685075320,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"调"},"index":0,"finish_reason":null}]} data: {"id":"chatcmpl-7KJM8RQ9fDHPB7WbhCevAGbbRRiGW","object":"chat.completion.chunk","created":1685075320,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"用"},"index":0,"finish_reason":null}]} data: {"id":"chatcmpl-7KJM8RQ9fDHPB7WbhCevAGbbRRiGW","object":"chat.completion.chunk","created":1685075320,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"次"},"index":0,"finish_reason":null}]} data: {"id":"chatcmpl-7KJM8RQ9fDHPB7WbhCevAGbbRRiGW","object":"chat.completion.chunk","created":1685075320,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"数"},"index":0,"finish_reason":null}]} data: {"id":"chatcmpl-7KJM8RQ9fDHPB7WbhCevAGbbRRiGW","object":"chat.completion.chunk","created":1685075320,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"和"},"index":0,"finish_reason":null}]} data: {"id":"chatcmpl-7KJM8RQ9fDHPB7WbhCevAGbbRRiGW","object":"chat.completion.chunk","created":1685075320,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"使用"},"index":0,"finish_reason":null}]} data: {"id":"chatcmpl-7KJM8RQ9fDHPB7WbhCevAGbbRRiGW","object":"chat.completion.chunk","created":1685075320,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"的"},"index":0,"finish_reason":null}]} data: {"id":"chatcmpl-7KJM8RQ9fDHPB7WbhCevAGbbRRiGW","object":"chat.completion.chunk","created":1685075320,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"API"},"index":0,"finish_reason":null}]} data: {"id":"chatcmpl-7KJM8RQ9fDHPB7WbhCevAGbbRRiGW","object":"chat.completion.chunk","created":1685075320,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"令"},"index":0,"finish_reason":null}]} data: {"id":"chatcmpl-7KJM8RQ9fDHPB7WbhCevAGbbRRiGW","object":"chat.completion.chunk","created":1685075320,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"牌"},"index":0,"finish_reason":null}]} data: {"id":"chatcmpl-7KJM8RQ9fDHPB7WbhCevAGbbRRiGW","object":"chat.completion.chunk","created":1685075320,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"。"},"index":0,"finish_reason":null}]} data: {"id":"chatcmpl-7KJM8RQ9fDHPB7WbhCevAGbbRRiGW","object":"chat.completion.chunk","created":1685075320,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"但"},"index":0,"finish_reason":null}]} data: {"id":"chatcmpl-7KJM8RQ9fDHPB7WbhCevAGbbRRiGW","object":"chat.completion.chunk","created":1685075320,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"是"},"index":0,"finish_reason":null}]} data: {"id":"chatcmpl-7KJM8RQ9fDHPB7WbhCevAGbbRRiGW","object":"chat.completion.chunk","created":1685075320,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"，"},"index":0,"finish_reason":null}]} data: {"id":"chatcmpl-7KJM8RQ9fDHPB7WbhCevAGbbRRiGW","object":"chat.completion.chunk","created":1685075320,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"我们"},"index":0,"finish_reason":null}]} data: {"id":"chatcmpl-7KJM8RQ9fDHPB7WbhCevAGbbRRiGW","object":"chat.completion.chunk","created":1685075320,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"不"},"index":0,"finish_reason":null}]} data: {"id":"chatcmpl-7KJM8RQ9fDHPB7WbhCevAGbbRRiGW","object":"chat.completion.chunk","created":1685075320,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"会"},"index":0,"finish_reason":null}]} data: {"id":"chatcmpl-7KJM8RQ9fDHPB7WbhCevAGbbRRiGW","object":"chat.completion.chunk","created":1685075320,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"存"},"index":0,"finish_reason":null}]} data: {"id":"chatcmpl-7KJM8RQ9fDHPB7WbhCevAGbbRRiGW","object":"chat.completion.chunk","created":1685075320,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"储"},"index":0,"finish_reason":null}]} data: {"id":"chatcmpl-7KJM8RQ9fDHPB7WbhCevAGbbRRiGW","object":"chat.completion.chunk","created":1685075320,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"或"},"index":0,"finish_reason":null}]} data: {"id":"chatcmpl-7KJM8RQ9fDHPB7WbhCevAGbbRRiGW","object":"chat.completion.chunk","created":1685075320,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"访"},"index":0,"finish_reason":null}]} data: {"id":"chatcmpl-7KJM8RQ9fDHPB7WbhCevAGbbRRiGW","object":"chat.completion.chunk","created":1685075320,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"问"},"index":0,"finish_reason":null}]} data: {"id":"chatcmpl-7KJM8RQ9fDHPB7WbhCevAGbbRRiGW","object":"chat.completion.chunk","created":1685075320,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"发送"},"index":0,"finish_reason":null}]} data: {"id":"chatcmpl-7KJM8RQ9fDHPB7WbhCevAGbbRRiGW","object":"chat.completion.chunk","created":1685075320,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"到"},"index":0,"finish_reason":null}]} data: {"id":"chatcmpl-7KJM8RQ9fDHPB7WbhCevAGbbRRiGW","object":"chat.completion.chunk","created":1685075320,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"API"},"index":0,"finish_reason":null}]} data: {"id":"chatcmpl-7KJM8RQ9fDHPB7WbhCevAGbbRRiGW","object":"chat.completion.chunk","created":1685075320,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"的"},"index":0,"finish_reason":null}]} data: {"id":"chatcmpl-7KJM8RQ9fDHPB7WbhCevAGbbRRiGW","object":"chat.completion.chunk","created":1685075320,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"请求"},"index":0,"finish_reason":null}]} data: {"id":"chatcmpl-7KJM8RQ9fDHPB7WbhCevAGbbRRiGW","object":"chat.completion.chunk","created":1685075320,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"内容"},"index":0,"finish_reason":null}]} data: {"id":"chatcmpl-7KJM8RQ9fDHPB7WbhCevAGbbRRiGW","object":"chat.completion.chunk","created":1685075320,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"。"},"index":0,"finish_reason":null}]} data: {"id":"chatcmpl-7KJM8RQ9fDHPB7WbhCevAGbbRRiGW","object":"chat.completion.chunk","created":1685075320,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{},"index":0,"finish_reason":"stop"}]} data: [DONE] ```
ripperhe commented 1 year ago

@liby 这个原因应该是代理服务器没处理好，把流式数据截断返回了。（把一条 data 从中间截断了）

Bob 只会把服务器返回的数据原封不动抛给插件，如果服务器已经截断了，Bob 这里返回的数据也是截断的。

从抓包 response 看不出来问题，因为被截断的流式数据最终会合并在一起展示，所以看不出来。但是从你日志可以看出来服务器将一条 data 分成了两部分返回。

可以参考下 #504，可能是同样的问题

image
liby commented 1 year ago

@ripperhe

看起来有这个可能，不过这个项目并没有使用 ningx 相关的显式配置，而是采用的 Vercel edge function 的默认配置。

尝试在其他桌面客户端比如 OpenCat、Openai Translator 上使用也没有遇到类似截断的问题，所以才猜测是不是 Bob 在处理数据时存在问题

ripperhe commented 1 year ago

@liby

这个出现概率高吗？

如果其他客户端都没有问题的话，我后面也检查下是不是 Bob 网络库的问题，不过这个可能较小。

liby commented 1 year ago

如果超过 3 个单词以上 80% 的几率会出现截断

ripperhe commented 1 year ago

@liby 考虑了下，难免会有截断的情况，参考 openai 官方代码做个兼容逻辑吧

https://github.com/openai/openai-cookbook/blob/main/apps/file-q-and-a/nextjs/src/services/openai.ts

image
liby commented 1 year ago

感谢提供思路，已解决

liby commented 11 months ago

@ripperhe 最近从同事那里得到反馈说这个问题又出现了，Debug 了一下，发现部署在 Vercel Edge function 上 的 OpenAI API 返回的流数据除了截断的问题以外，还有顺序随机的问题（

比如我尝试翻译："Too easy! This will connect to the OpenAI API and return the response to the client. The only thing we need to do now is modify the response to be a stream." 得到结果：「OpenAI API并将响。现在应为流。」 这时候查看日志：

info    18:28:26.676511+0800    Bob [I] [yetone.openai.translator]: index 1: data: {"id":"chatcmpl-8QYZPlxerdJll72kSFpO2rPVDBO9x","object":"chat.completion.chunk","created":1701340107,"model":"gpt-3.5-turbo-0301","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":"Open"},"finish_reason":null}]}

data: {"id":"chatcmpl-8QYZPlxerdJll72kSFpO2rPVDBO9x","object":"chat.completion.chunk","created":1701340107,"model":"gpt-3.5-turbo-0301","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":"AI"},"finish_reason":null}]}

data: {"id":"chatcmpl-8QYZPlxerdJll72kSFpO2rPVDBO9x","object":"chat.completion.chunk","created":1701340107,"model":"gpt-3.5-turbo-0301","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":" A
info    18:28:26.717190+0800    Bob [I] [yetone.openai.translator]: index 2: PI"},"finish_reason":null}]}

data: {"id":"chatcmpl-8QYZPlxerdJll72kSFpO2rPVDBO9x","object":"chat.completion.chunk","created":1701340107,"model":"gpt-3.5-turbo-0301","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":"并"},"finish_reason":null}]}

data: {"id":"chatcmpl-8QYZPlxerdJll72kSFpO2rPVDBO9x","object":"chat.completion.chunk","created":1701340107,"model":"gpt-3.5-turbo-0301","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":"将"},"finish_reason":null}]}

data: {"id":"chatcmpl-8QYZPlxerdJll72kSFpO2rPVDBO9x","object":"chat.completion.chunk","created":1701340107,"model":"gpt-3.5-turbo-0301","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":"响
info    18:28:26.771152+0800    Bob [I] [yetone.openai.translator]: index 3: "},"finish_reason":null}]}
info    18:28:26.887237+0800    Bob [I] [yetone.openai.translator]: index 4: data: {"id":"chatcmpl-8QYZP
info    18:28:26.887370+0800    Bob [I] [yetone.openai.translator]: index 5: lxerdJll72kSFpO2rPVDBO9x","object":"chat.completion.chunk","created":1701340107,"model":"gpt-3.5-turbo-0301","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":"。"},"finish_reason":null}]}

data: {"id":"chatcmpl-8QYZPlxerdJll72kSFpO2rPVDBO9x","object":"chat.completion.chunk","created":1701340107,"model":"gpt-3.5-turbo-0301","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":"现"},"finish_reason":null}]}

data: {"id":"chatcmpl-8QYZPlxerdJll72kSFpO2rPVDBO9x","object":"chat.completion.chunk","created":1701340107,"model":"gpt-3.5-turbo-0301","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":"在
info    18:28:26.960846+0800    Bob [I] [yetone.openai.translator]: index 6: "},"finish_reason":null}]}
info    18:28:27.145579+0800    Bob [I] [yetone.openai.translator]: index 7: 响"},"finish_reason":null}]}

data: {"id":"chatcmpl-8QYZPlxerdJll72kSFpO2rPVDBO9x","object":"chat.completion.chunk","created":1701340107,"model":"gpt-3.5-turbo-0301","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":"应"},"finish_reason":null}]}
info    18:28:27.145919+0800    Bob [I] [yetone.openai.translator]: index 8: data: {"id":"chatcmpl-8QYZPlxerdJll72kSFpO2rPVDBO9x","object":"chat.completion.chunk","created":1701340107,"model":"gpt-3.5-turbo-0301","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":"为"},"finish_reason":null}]}

data: {"id":"chatcmpl-8QYZPlxerdJll72kSFpO2rPVDBO9x","object":"chat.completion.chunk","created":1701340107,"model":"gpt-3.5-turbo-0301","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":"流"},"finish_reason":null}]}

data: {"id":"chatcmpl-8QYZPlxerdJll72kSFpO2rPVDBO9x","object":"chat.completion.chunk","created":1701340107,"model":"gpt-3.5-turbo-0301","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":"
info    18:28:27.146450+0800    Bob [I] [yetone.openai.translator]: index 9: 。"},"finish_reason":null}]}

data: {"id":"chatcmpl-8QYZPlxerdJll72kSFpO2rPVDBO9x","object":"chat.completion.chunk","created":1701340107,"model":"gpt-3.5-turbo-0301","system_fingerprint":null,"choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: [DONE]

而且这个问题还不是稳定复现的；且有些时候还会出现在数据流中间就会出现 data: [DONE] 数据的情况（概率极低）。

我 Debug 了一天，没有什么头绪，想请教一下作者有什么好的建议嘛 😢

ripperhe commented 11 months ago

@ripperhe 最近从同事那里得到反馈说这个问题又出现了，Debug 了一下，发现部署在 Vercel Edge function 上 的 OpenAI API 返回的流数据除了截断的问题以外，还有顺序随机的问题（

比如我尝试翻译："Too easy! This will connect to the OpenAI API and return the response to the client. The only thing we need to do now is modify the response to be a stream." 得到结果：「OpenAI API并将响。现在应为流。」 这时候查看日志：

info  18:28:26.676511+0800    Bob [I] [yetone.openai.translator]: index 1: data: {"id":"chatcmpl-8QYZPlxerdJll72kSFpO2rPVDBO9x","object":"chat.completion.chunk","created":1701340107,"model":"gpt-3.5-turbo-0301","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":"Open"},"finish_reason":null}]}

data: {"id":"chatcmpl-8QYZPlxerdJll72kSFpO2rPVDBO9x","object":"chat.completion.chunk","created":1701340107,"model":"gpt-3.5-turbo-0301","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":"AI"},"finish_reason":null}]}

data: {"id":"chatcmpl-8QYZPlxerdJll72kSFpO2rPVDBO9x","object":"chat.completion.chunk","created":1701340107,"model":"gpt-3.5-turbo-0301","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":" A
info  18:28:26.717190+0800    Bob [I] [yetone.openai.translator]: index 2: PI"},"finish_reason":null}]}

data: {"id":"chatcmpl-8QYZPlxerdJll72kSFpO2rPVDBO9x","object":"chat.completion.chunk","created":1701340107,"model":"gpt-3.5-turbo-0301","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":"并"},"finish_reason":null}]}

data: {"id":"chatcmpl-8QYZPlxerdJll72kSFpO2rPVDBO9x","object":"chat.completion.chunk","created":1701340107,"model":"gpt-3.5-turbo-0301","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":"将"},"finish_reason":null}]}

data: {"id":"chatcmpl-8QYZPlxerdJll72kSFpO2rPVDBO9x","object":"chat.completion.chunk","created":1701340107,"model":"gpt-3.5-turbo-0301","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":"响
info  18:28:26.771152+0800    Bob [I] [yetone.openai.translator]: index 3: "},"finish_reason":null}]}
info  18:28:26.887237+0800    Bob [I] [yetone.openai.translator]: index 4: data: {"id":"chatcmpl-8QYZP
info  18:28:26.887370+0800    Bob [I] [yetone.openai.translator]: index 5: lxerdJll72kSFpO2rPVDBO9x","object":"chat.completion.chunk","created":1701340107,"model":"gpt-3.5-turbo-0301","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":"。"},"finish_reason":null}]}

data: {"id":"chatcmpl-8QYZPlxerdJll72kSFpO2rPVDBO9x","object":"chat.completion.chunk","created":1701340107,"model":"gpt-3.5-turbo-0301","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":"现"},"finish_reason":null}]}

data: {"id":"chatcmpl-8QYZPlxerdJll72kSFpO2rPVDBO9x","object":"chat.completion.chunk","created":1701340107,"model":"gpt-3.5-turbo-0301","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":"在
info  18:28:26.960846+0800    Bob [I] [yetone.openai.translator]: index 6: "},"finish_reason":null}]}
info  18:28:27.145579+0800    Bob [I] [yetone.openai.translator]: index 7: 响"},"finish_reason":null}]}

data: {"id":"chatcmpl-8QYZPlxerdJll72kSFpO2rPVDBO9x","object":"chat.completion.chunk","created":1701340107,"model":"gpt-3.5-turbo-0301","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":"应"},"finish_reason":null}]}
info  18:28:27.145919+0800    Bob [I] [yetone.openai.translator]: index 8: data: {"id":"chatcmpl-8QYZPlxerdJll72kSFpO2rPVDBO9x","object":"chat.completion.chunk","created":1701340107,"model":"gpt-3.5-turbo-0301","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":"为"},"finish_reason":null}]}

data: {"id":"chatcmpl-8QYZPlxerdJll72kSFpO2rPVDBO9x","object":"chat.completion.chunk","created":1701340107,"model":"gpt-3.5-turbo-0301","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":"流"},"finish_reason":null}]}

data: {"id":"chatcmpl-8QYZPlxerdJll72kSFpO2rPVDBO9x","object":"chat.completion.chunk","created":1701340107,"model":"gpt-3.5-turbo-0301","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":"
info  18:28:27.146450+0800    Bob [I] [yetone.openai.translator]: index 9: 。"},"finish_reason":null}]}

data: {"id":"chatcmpl-8QYZPlxerdJll72kSFpO2rPVDBO9x","object":"chat.completion.chunk","created":1701340107,"model":"gpt-3.5-turbo-0301","system_fingerprint":null,"choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: [DONE]

而且这个问题还不是稳定复现的；且有些时候还会出现在数据流中间就会出现 data: [DONE] 数据的情况（概率极低）。

我 Debug 了一天，没有什么头绪，想请教一下作者有什么好的建议嘛 😢

所以服务器返回的数据就是乱的对吧，而且每一段数据里面也没有可以用来排序的参数，我也没啥思路 😂，有见到其他 App 有这种问题吗？

有没有抓包验证过 Bob 日志和服务器实际返回的顺序是一致的，不过 Bob 用的是主流的网络框架，应该不存在把顺序搞乱这种问题。

liby commented 11 months ago

有没有抓包验证过 Bob 日志和服务器实际返回的顺序是一致的，不过 Bob 用的是主流的网络框架，应该不存在把顺序搞乱这种问题

Charles 抓包看了没问题，不过就像你之前讲过的，因为截断的流式数据最终会合并在一起展示，所以抓包工具看不出来问题也是正常的。

但是从 Console 输出的 log（在代码里手动打印返回的 streamData.text）能看出来，部署在 Vercel Edge function 的 OpenAI API 返回的数据就是有被随机打乱和截断的问题，应该跟 Bob 无关。

顺着这个问题我找到了一个 issue：中文字符截断导致乱码问题，虽然问题导致的结果不同，但是导致问题的原因是高度相似的，我还试了一下，中译英返回的流数据也存在被截断的情况，但尝试了很多次都没有流数据乱序的问题，翻译结果也是对的。

比如我将「有没有抓包验证过 Bob 日志和服务器实际返回的顺序是一致的，不过 Bob 用的是主流的网络框架，应该不存在把顺序搞乱这种问题。」翻译成英文，在 Console 中看到的日志：

info    10:02:27.317675+0800    Bob [I] [yetone.openai.translator]: index 0: data: {"id":"chatcmpl-8Qn9JflcINoizIE7KF69aGcuh44Md","object":"chat.completion.chunk","created":1701396149,"model":"gpt-3.5-turbo-1106","system_fingerprint":"fp_eeff13170a","choices":[{"index":0,"delta":{"role":"assistant","content":""},"finish_reason":null}]}

data: {"id":"chatcmpl-8Qn9JflcINoizIE7KF69aGcuh44Md","object":"chat.completion.chunk","created":1701396149,"model":"gpt-3.5-turbo-1106","system_fingerprint":"fp_eeff13170a","choices":[{"index":0,"delta":{"content":"Have"},"finish_reason":null}]}

data: {"id":"chatcmpl-8Qn9JflcINoizIE7KF69aGcuh44Md","object":"chat.completion.chunk","created":1701396149,"model":"gpt-3.5-turbo-1106","system_fingerprint":"fp_eeff13170a","choices":[{"index":0,"delta":{"content":" you"},"finish_reason":null}]}

data: {"id":"chatcmpl-8Qn9JflcINoizIE7KF69aGcuh44Md","object":"chat.completion.chunk","created":1701396149,"model":"gpt-3.5-turbo-1106","system_fingerprint":"fp_eeff13170a","choices":[{"index":0,"delta":{"content":" ve<…>
info    10:02:27.392592+0800    Bob [I] [yetone.openai.translator]: index 1: data: {"id":"chatcmpl-8Qn9J
info    10:02:27.392694+0800    Bob [I] [yetone.openai.translator]: index 2: flcINoizIE7KF69aGcuh44Md","object":"chat.completion.chunk","created":1701396149,"model":"gpt-3.5-turbo-1106","system_fingerprint":"fp_eeff13170a","choices":[{"index":0,"delta":{"content":" that"},"finish_reason":null}]}

data: {"id":"chatcmpl-8Qn9JflcINoizIE7KF69aGcuh44Md","object":"chat.completion.chunk","created":1701396149,"model":"gpt-3.5-turbo-1106","system_fingerprint":"fp_eeff13170a","choices":[{"index":0,"delta":{"content":" the"},"finish_reason":null}]}

data: {"id":"chatcmpl-8Qn9JflcINoizIE7KF69aGcuh44Md","object":"chat.completion.chunk","created":1701396149,"model":"gpt-3.5-turbo-1106","system_fingerprint":"fp_eeff13170a","choices":[{"index":0,"delta":{"content":" order"},"finish_reason":null}]}

data: {"id":"chatcmpl-8Qn9JflcINoizIE7KF69aGcuh44Md","object":"chat.completion.chunk","created":1701396149,"model":"gpt-3.5-turbo-1106","system_fingerprint":"fp_eeff13170a","choices":[{"index":0,"delta":{"content":" of"},"finish_reason":null}]}

data: {"id"<…>
info    10:02:27.438997+0800    Bob [I] [yetone.openai.translator]: index 3: "},"finish_reason":null}]}

data: {"id":"chatcmpl-8Qn9JflcINoizIE7KF69aGcuh44Md","object":"chat.completion.chunk","created":1701396149,"model":"gpt-3.5-turbo-1106","system_fingerprint":"fp_eeff13170a","choices":[{"index":0,"delta":{"content":" logs"},"finish_reason":null}]}

data: {"id":"chatcmpl-8Qn9JflcINoizIE7KF69aGcuh44Md","object":"chat.completion.chunk","created":1701396149,"model":"gpt-3.5-turbo-1106","system_fingerprint":"fp_eeff13170a","choices":[{"index":0,"delta":{"content":" and"},"finish_reason":null}]}

data: {"id":"chatcmpl-8Qn9JflcINoizIE7KF69aGcuh44Md","object":"chat.completion.chunk","created":1701396149,"model":"gpt-3.5-turbo-1106","system_fingerprint":"fp_eeff13170a","choices":[{"index":0,"delta":{"content":" th
info    10:02:27.492865+0800    Bob [I] [yetone.openai.translator]: index 4: e"},"finish_reason":null}]}

data: {"id":"chatcmpl-8Qn9JflcINoizIE7KF69aGcuh44Md","object":"chat.completion.chunk","created":1701396149,"model":"gpt-3.5-turbo-1106","system_fingerprint":"fp_eeff13170a","choices":[{"index":0,"delta":{"content":" actual"},"finish_reason":null}]}

data: {"id":"chatcmpl-8Qn9JflcINoizIE7KF69aGcuh44Md","object":"chat.completion.chunk","created":1701396149,"model":"gpt-3.5-turbo-1106","system_fingerprint":"fp_eeff13170a","choices":[{"index":0,"delta":{"content":" server"},"finish_reason":null}]}

data: {"id":"chatcmpl-8Qn9JflcINoizIE7KF69aGcuh44Md","object":"chat.completion.chunk","created":1701396149,"model":"gpt-3.5-turbo-1106","system_fingerprint":"fp_eeff13170a","choices":[{"index":0,"delta":{"content":" response
info    10:02:27.585989+0800    Bob [I] [yetone.openai.translator]: index 5: s"},"finish_reason":null}]}
info    10:02:27.586229+0800    Bob [I] [yetone.openai.translator]: index 6: data: {"id":"chatcmpl-8Qn9JflcINoizIE7KF69aGcuh44Md","object":"chat.completion.chunk","created":1701396149,"model":"gpt-3.5-turbo-1106","system_fingerprint":"fp_eeff13170a","choices":[{"index":0,"delta":{"content":" are"},"finish_reason":null}]}

data: {"id":"chatcmpl-8Qn9JflcINoizIE7KF69aGcuh44Md","object":"chat.completion.chunk","created":1701396149,"model":"gpt-3.5-turbo-1106","system_fingerprint":"fp_eeff13170a","choices":[{"index":0,"delta":{"content":" consistent"},"finish_reason":null}]}

data: {"id":"chatcmpl-8Qn9JflcINoizIE7KF69aGcuh44Md","object":"chat.completion.chunk","created":1701396149,"model":"gpt-3.5-turbo-1106","system_fingerprint":"fp_eeff13170a","choices":[{"index":0,"delta":{"content":" by
info    10:02:27.618299+0800    Bob [I] [yetone.openai.translator]: index 7: "},"finish_reason":null}]}

data: {"id":"chatcmpl-8Qn9JflcINoizIE7KF69aGcuh44Md","object":"chat.completion.chunk","created":1701396149,"model":"gpt-3.5-turbo-1106","system_fingerprint":"fp_eeff13170a","choices":[{"index":0,"delta":{"content":" capturing"},"finish_reason":null}]}

data: {"id":"chatcmpl-8Qn9JflcINoizIE7KF69aGcuh44Md","object":"chat.completion.chunk","created":1701396149,"model":"gpt-3.5-turbo-1106","system_fingerprint":"fp_eeff13170a","choices":[{"index":0,"delta":{"content":" packets"},"finish_reason":null}]}

data: {"id":"chatcmpl-8Qn9JflcINoizIE7KF69aGcuh44Md","object":"chat.completion.chunk","created":1701396149,"model":"gpt-3.5-turbo-1106","system_fingerprint":"fp_eeff13170a","choices":[{"index":0,"delta":{"content":"?
info    10:02:27.652914+0800    Bob [I] [yetone.openai.translator]: index 8: "},"finish_reason":null}]}

data: {"id":"chatcmpl-8Qn9JflcINoizIE7KF69aGcuh44Md","object":"chat.completion.chunk","created":1701396149,"model":"gpt-3.5-turbo-1106","system_fingerprint":"fp_eeff13170a","choices":[{"index":0,"delta":{"content":" However"},"finish_reason":null}]}

data: {"id":"chatcmpl-8Qn9JflcINoizIE7KF69aGcuh44Md","object":"chat.completion.chunk","created":1701396149,"model":"gpt-3.5-turbo-1106","system_fingerprint":"fp_eeff13170a","choices":[{"index":0,"delta":{"content":","},"finish_reason":null}]}

data: {"id":"chatcmpl-8Qn9JflcINoizIE7KF69aGcuh44Md","object":"chat.completion.chunk","created":1701396149,"model":"gpt-3.5-turbo-1106","system_fingerprint":"fp_eeff13170a","choices":[{"index":0,"delta":{"content":"
info    10:02:27.722039+0800    Bob [I] [yetone.openai.translator]: index 9: Bob"},"finish_reason":null}]}
info    10:02:27.722357+0800    Bob [I] [yetone.openai.translator]: index 10: data: {"id":"chatcmpl-8Qn9JflcINoizIE7KF69aGcuh44Md","object":"chat.completion.chunk","created":1701396149,"model":"gpt-3.5-turbo-1106","system_fingerprint":"fp_eeff13170a","choices":[{"index":0,"delta":{"content":" is"},"finish_reason":null}]}

data: {"id":"chatcmpl-8Qn9JflcINoizIE7KF69aGcuh44Md","object":"chat.completion.chunk","created":1701396149,"model":"gpt-3.5-turbo-1106","system_fingerprint":"fp_eeff13170a","choices":[{"index":0,"delta":{"content":" using"},"finish_reason":null}]}

data: {"id":"chatcmpl-8Qn9JflcINoizIE7KF69aGcuh44Md","object":"chat.completion.chunk","created":1701396149,"model":"gpt-3.5-turbo-1106","system_fingerprint":"fp_eeff13170a","choices":[{"index":0,"delta":{"content":" a
info    10:02:27.731580+0800    Bob [I] [yetone.openai.translator]: index 11: "},"finish_reason":null}]}

data: {"id":"chatcmpl-8Qn9JflcINoizIE7KF69aGcuh44Md","object":"chat.completion.chunk","created":1701396149,"model":"gpt-3.5-turbo-1106","system_fingerprint":"fp_eeff13170a","choices":[{"index":0,"delta":{"content":" mainstream"},"finish_reason":null}]}

data: {"id":"chatcmpl-8Qn9JflcINoizIE7KF69aGcuh44Md","object":"chat.completion.chunk","created":1701396149,"model":"gpt-3.5-turbo-1106","system_fingerprint":"fp_eeff13170a","choices":[{"index":0,"delta":{"content":" network"},"finish_reason":null}]}

data: {"id":"chatcmpl-8Qn9JflcINoizIE7KF69aGcuh44Md","object":"chat.completion.chunk","created":1701396149,"model":"gpt-3.5-turbo-1106","system_fingerprint":"fp_eeff13170a","choices":[{"index":0,"delta":{"content":" frame
info    10:02:27.771918+0800    Bob [I] [yetone.openai.translator]: index 12: work"},"finish_reason":null}]}
info    10:02:27.772186+0800    Bob [I] [yetone.openai.translator]: index 13: data: {"id":"chatcmpl-8Qn9JflcINoizIE7KF69aGcuh44Md","object":"chat.completion.chunk","created":1701396149,"model":"gpt-3.5-turbo-1106","system_fingerprint":"fp_eeff13170a","choices":[{"index":0,"delta":{"content":","},"finish_reason":null}]}

data: {"id":"chatcmpl-8Qn9JflcINoizIE7KF69aGcuh44Md","object":"chat.completion.chunk","created":1701396149,"model":"gpt-3.5-turbo-1106","system_fingerprint":"fp_eeff13170a","choices":[{"index":0,"delta":{"content":" so"},"finish_reason":null}]}

data: {"id":"chatcmpl-8Qn9JflcINoizIE7KF69aGcuh44Md","object":"chat.completion.chunk","created":1701396149,"model":"gpt-3.5-turbo-1106","system_fingerprint":"fp_eeff13170a","choices":[{"index":0,"delta":{"content":" the
info    10:02:27.829998+0800    Bob [I] [yetone.openai.translator]: index 14: re"},"finish_reason":null}]}

data: {"id":"chatcmpl-8Qn9JflcINoizIE7KF69aGcuh44Md","object":"chat.completion.chunk","created":1701396149,"model":"gpt-3.5-turbo-1106","system_fingerprint":"fp_eeff13170a","choices":[{"index":0,"delta":{"content":" should"},"finish_reason":null}]}

data: {"id":"chatcmpl-8Qn9JflcINoizIE7KF69aGcuh44Md","object":"chat.completion.chunk","created":1701396149,"model":"gpt-3.5-turbo-1106","system_fingerprint":"fp_eeff13170a","choices":[{"index":0,"delta":{"content":" be"},"finish_reason":null}]}

data: {"id":"chatcmpl-8Qn9JflcINoizIE7KF69aGcuh44Md","object":"chat.completion.chunk","created":1701396149,"model":"gpt-3.5-turbo-1106","system_fingerprint":"fp_eeff13170a","choices":[{"index":0,"delta":{"content":" n
info    10:02:27.870390+0800    Bob [I] [yetone.openai.translator]: index 15: o"},"finish_reason":null}]}

data: {"id":"chatcmpl-8Qn9JflcINoizIE7KF69aGcuh44Md","object":"chat.completion.chunk","created":1701396149,"model":"gpt-3.5-turbo-1106","system_fingerprint":"fp_eeff13170a","choices":[{"index":0,"delta":{"content":" problem"},"finish_reason":null}]}

data: {"id":"chatcmpl-8Qn9JflcINoizIE7KF69aGcuh44Md","object":"chat.completion.chunk","created":1701396149,"model":"gpt-3.5-turbo-1106","system_fingerprint":"fp_eeff13170a","choices":[{"index":0,"delta":{"content":" with"},"finish_reason":null}]}

data: {"id":"chatcmpl-8Qn9JflcINoizIE7KF69aGcuh44Md","object":"chat.completion.chunk","created":1701396149,"model":"gpt-3.5-turbo-1106","system_fingerprint":"fp_eeff13170a","choices":[{"index":0,"delta":{"content":" mes
info    10:02:27.896538+0800    Bob [I] [yetone.openai.translator]: index 16: sing"},"finish_reason":null}]}
info    10:02:27.896716+0800    Bob [I] [yetone.openai.translator]: index 17: data: {"id":"chatcmpl-8Qn9JflcINoizIE7KF69aGcuh44Md","object":"chat.completion.chunk","created":1701396149,"model":"gpt-3.5-turbo-1106","system_fingerprint":"fp_eeff13170a","choices":[{"index":0,"delta":{"content":" up"},"finish_reason":null}]}

data: {"id":"chatcmpl-8Qn9JflcINoizIE7KF69aGcuh44Md","object":"chat.completion.chunk","created":1701396149,"model":"gpt-3.5-turbo-1106","system_fingerprint":"fp_eeff13170a","choices":[{"index":0,"delta":{"content":" the"},"finish_reason":null}]}

data: {"id":"chatcmpl-8Qn9JflcINoizIE7KF69aGcuh44Md","object":"chat.completion.chunk","created":1701396149,"model":"gpt-3.5-turbo-1106","system_fingerprint":"fp_eeff13170a","choices":[{"index":0,"delta":{"content":" order"},"finish_reason":null}]}

data: {"id":"chatcmpl-8Qn9JflcINoizIE7KF69aGcuh44Md","object":"chat.completion.chunk","created":1701396149,"model":"gpt-3.5-turbo-1106","system_fingerprint":"fp_eeff13170a","choices":[{"index":0,"delta":{"content":"."},"finish_reas<…>
info    10:02:27.896998+0800    Bob [I] [yetone.openai.translator]: index 18:

虽然有些 log 被省略展示了，甚至最后也没有看到 data: [DONE]，但是翻译结果是没问题的：

image
ripperhe commented 11 months ago

确实，抓包结果里面数据最终会合并在一起，以致于看不出来数据传输过程是否有「截断」，不过数据返回的顺序都有问题的话，那我理解抓包里面的结果合并起来之后也应该是错乱的。

那目前看起来「截断」能兼容，但是「顺序错乱」无解。我也没啥思路，后面有啥思路再同步给你。