-
## description
Put an option to edit the original text in the Editor so that the annotator can edit the text when needed
## steps
- [x] add Edit option on the tex
-
### Description:
Create a series of scripts to validate word segmentation by ensuring that each word in the target field:
- Exists in the unique word list.
- Is neither oversegmented (unnecessari…
-
[WB3c](https://www.unicode.org/reports/tr29/#WB3c) and [WB3c](https://www.unicode.org/reports/tr29/#WB3c) interact in the same way [LB8a](https://www.unicode.org/reports/tr14/#LB8a) and [LB9](https://…
-
### Description
Word segmentation being the center of many NLP tasks but yet we still havent got a AI based word segmenter but things have changed not as we have the dataset and the pretrained model …
-
I am using version 1.10.1 of meisearch to test Chinese search and have encountered the following issues:
The test code is as follows:
```
import com.alibaba.fastjson.JSON;
import com.alibaba.f…
-
Hi there! I'm having trouble using the Jieba library for word segmentation in the dify-sandbox environment. When I run the code below, I get the error message: "Building prefix dict from the default d…
-
I've tried `segmentation-lemma-tagging/run_inf.py` with various modes on the following sentence:
> āsīdaśeṣanarapatiśiraḥsamarcitaśāsanaḥ pākaśāsana ivāparacaturudadhimālāmekhalāyā bhuvo bhartā pra…
-
**分词**
- [中文分词文献列表](http://zhangkaixu.github.io/bibpage/cws.html)
- 张开旭, Kaixu Zhang
- 陈新驰
- [深度学习中文分词调研](http://www.hankcs.com/nlp/segment/depth-learning-chinese-word-segmentation-survey.html)
…
-
### Self Checks
- [X] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones.
- [X] I confirm that I am using English to…
-
**Description**
We want all the opensource Tibetan word segmented data and save it in a standard format.
The format should be:
```
[
{
'source': 'བོད་ཀྱི་གླུ་གར་རོལ་དབྱངས་ལ་གཞི་རྩའི་ཐོག་ནས་དབྱ…