Open libukai opened 1 week ago
There are similar issues and discussions related to the language setting in the Question & Answer segmentation format not working as expected:
An open issue titled "Q&A format choose Chinese. Shows 0 paragraphs. Other languages work normally." suggests that the problem might be caused by the _format_split_text
and format_split_text
functions. If the input text contains special characters like "*", these functions may return an empty list, resulting in an empty qa_preview
. Additionally, if the output is not in the expected format, the qa_preview
will also be empty.
A closed issue titled "Adopt Q&A segmentation mode, segmentation use option: simplified Chinese, or a large number of English segments will be generated." indicates that testing with OpenAI GPT-3.5 worked fine. It was suggested to emphasize the use of language above the prompt and to check out the latest code for improvements.
These references might provide insights or solutions to the issue you're experiencing with the language setting in the segmentation format [1][2].
To continue talking to Dosu, mention @dosu.
None of the issues provide useful info. I changed the model and it still generated English segments.
And the language setting auto back to English after the process finished.
Self Checks
Dify version
0.9.2
Cloud or Self Hosted
Self Hosted (Docker)
Steps to reproduce
Even the language is set to Chinese Simplified, the result segment of the index is English.
✔️ Expected Behavior
No response
❌ Actual Behavior
No response