-
Look for instances where splitting text with .split(" "), tokenize with nltk instead or figure out how to tokenize with more than whitespace
-
### System Info
peft==0.13.0
### Who can help?
@EricLBuehler
### Information
- [ ] The official example scripts
- [X] My own modified scripts
### Tasks
- [ ] An officially supported task in th…
-
Would consider release the training code?
-
您好,请问是哪里出现了空行呢
-
I encountered a core dump while fuzzing that traces back to jsfuzz's `versifier.js:413`
```
[6242:0x34faa20] 18540 ms: Mark-sweep 1291.7 (1426.7) -> 1291.7 (1427.2) MB, 15.0 / 0.0 ms (average …
-
### Feature request
The OpenAI API `/embedding` endpoint accepts input for both text (list of strings) and [tokenized input](https://github.com/openai/openai-openapi/blob/893ba52242dbd5387a97b96444…
slyt updated
4 months ago
-
Hello,
I just wanted to know how can I do the Payment(Authorization) through the customer Payment profile?.
Also, how can I get the Payment profile token.
It's good to have some example with …
-
```
print(T.word_tokenize("xin chào, tôi là người Việt Nam", tokenize_option=0))
```
What `tokenize_option` available can I pass into tokenize_option.
-
### Reproducible in vscode.dev or in VS Code Desktop?
- [X] Not reproducible in [vscode.dev](https://vscode.dev) or VS Code Desktop
### Reproducible in the monaco editor playground?
- [ ] Not repro…
-