-
### Checked other resources
- [X] I added a very descriptive title to this issue.
- [X] I searched the LangChain.js documentation with the integrated search.
- [X] I used the GitHub search to fin…
-
Hello,
Trying to run tiktokenizer from the app folder and I'm getting the following error:
```
Error: Element type is invalid. Received a promise that resolves to: [object Promise]. Lazy elemen…
-
## What would you like to be added:
It would be great to generate/load encoder from tokenizer.json file like
https://huggingface.co/CohereForAI/aya-101/resolve/main/tokenizer.json
or
https://…
-
when Running the tests on travis-ci using Linux Xenial Ruby 3.0/1 AMD64 I get
```
Could not find gems matching 'tiktoken_ruby' valid for all resolution platforms
(aarch64-linux, arm-linux, arm64-d…
-
pip install chromadb --no-cache-dir
pip install tiktoken --no-cache-dir
安装完这两个依赖后 降低一下urllib3的版本 再配置一下终端的代理 就可以使用了
_Originally posted by @hjhjbmy in https://github.com/DDizzzy79/Ret…
-
### Describe the issue
Due to slow network, LLMLingua failed to download the model from huggingface during running.
```
requests.exceptions.ReadTimeout: (ReadTimeoutError("HTTPSConnectionPool(host…
-
`local ok, core = pcall(require, "tiktoken_core")`
https://github.com/yetone/avante.nvim/blob/26748729455cfc95704c204ec0d6cb548d99f493/lua/avante/tiktoken.lua#L53
The load is failed, and shown `…
-
In the current implementation for TotalTokens in ChatCompletion.
1. Initialise the chat state with PUT method
```
PUT http://localhost:7071/api/chats/test123
Content-Type: applicatio…
-
**Is your feature request related to a problem? Please describe.**
Transformers that use LLM calls suffer from overloading their endpoints, resulting in errors like these:
```
2024-07-10T17:47:10…
tinco updated
1 month ago
-
Mistral and Llama2 models are working nicely, but I cannot get Yi models to work with tokenization.
I tried many repos with original Yi and also with finetunes.
Error I get:
```
raise TokenizerN…