-
hello, I want to keep uppercase letter。 like example：
```func Test_Tokenize(t *testing.T) {
text := "Hello world, Helloworld. Winter is coming! 你好世界."
jieba := new(gse.Segmenter)
jieba.LoadDict…
-
```
On: 31/08/2017 -2.04
CARD PAYMENT TO SHELL TOTHILL,2.04 GBP, RATE 1.00/GBP ON 29-08-2013
My guess is:
> 6
Traceback (most recent call last):
File "/Users/arturo/Documents/GitHub/BankClas…
-
**Description**: Create a function that uses the trigram model from Task 1 to generate synthetic text sequences, starting with an initial two-character seed.
**Checklist**:
- [ ] Research methods …
-
Hello, i met the following issue after chatting with the localGPT for several rounds:
"llama_tokenize_with_model: too many tokens".
could you please hlep to check this?
appreciated!!!
-
Encountered an issue on executing the PayeezyTest.php file because Payeezy.php does not have the function tokenize().
-
Since the command meanings are exactly the same, command input should be case insensitive.
Example:
Add vs ADD
exit vs EXIT
In the above image, users will be confused as to why command i…
-
Hey,
I want to train a Tokenizer that operates on a custom PreTokenizer. I tried a mix of [this documentation post](https://huggingface.co/docs/tokenizers/pipeline) and [this example](https://githu…
-
Hi,
pack_install(tokenize).
only gave an option to use the .zip. When I tried, it failed because of missing libraries on my mac. Is there a list of dependencies I should install first?
Thanks.
-
I’m using dynamic mapping to ingest a JSON object with a field of an array of JSON objects. If the array element has a field with value that has -, _, #, etc. delimiters in it, e.g. a uuid, querying a…
-
### Description of the issue
**Description:** When running the [Jujutsu](https://github.com/martinvonz/jj) test suite, I'm seeing a flaky test I would like to skip. I tried using the following comman…