-
This is an error case not to forget that causes some trouble with the sentence segmentation.
The document is not CC-BY, referenced here: https://dx.doi.org/10.1063/1.1874292
Here the `delinquent`…
-
It would be neat to add some sort of sentence segmentation to the query time text analysis to trigger a break in tagging. For example (a very silly one!) the input document text is:
" I want to buy s…
-
NLTK version 3.8.2 changed the data format of the tokenizers from pickle to text files in order to patch a vulnerability (CVE-2024-39705).
Here's the PR in the nltk repo:
https://github.com/nltk/n…
-
first as on https://ichi.moe
- https://readevalprint.tumblr.com/post/97467849358/who-needs-graph-theory-anyway
- https://github.com/tshatrov/ichiran
or https://jisho.org
-
We will need to be able to extract all sentences that use the word *Galaxy* from an input document. This implies that we are able to split an input document on sentence boundaries.
NLTK will be su…
-
Ex. 1: (filename: `What the Panama Papers Reveal About the Art Market - The New York Times.txt`)
`Q: All necessary financial disclosures were made at the time of sale." The International Consortium of…
-
Sometimes the segmentation of a sentence into words is not optimal.
Allow the user to improve it.
Ideas:
- try another segmentation library: https://chatgpt.com/share/c5d8b072-8531-4395-8566-ae…
-
我在进行使用时,遇到
from FlagEmbedding import FlagModel
zsh: segmentation fault python
zsh: segmentation fault python eval_C-MTEB.py
这两个包均不能使用,sentence_transformers在进行使用时也会报错,但进行修改降低版本为2.2.2后可进行使用,
希望…
-
Apostrophes `ʼ` are not parsed correctly - sometimes they appear in pairs to mark quotations. The second apostrophe usually gets assigned to the following sentence and if there is none (-> end of chap…
-
I think it would be good to have better support for other languages where the concept of sentence and word boundaries vary. Implementing word-based movements equivalent to the current ones would be he…