-
Does anyone know if a java alternative exists?
-
It would be more intuitive for users if they can see how many tokens they have used.
> To see how many tokens are in a text string without making an API call, use OpenAI’s [tiktoken](https://github…
-
在pdm run build后,执行dist 目录下的 VectorVein.exe 命令后,报如下错误。
![image](https://github.com/AndersonBY/vector-vein/assets/45526589/ff4a46dd-54f7-42da-82ba-f8cbca6f8150)
这是dist/vector-vein目录下的文件列表:
目录: D:…
-
java.lang.IllegalStateException: Could not load /com/knuddels/jtokkit/r50k_base.tiktoken from resources
at com.knuddels.jtokkit.EncodingFactory.loadMergeableRanks(EncodingFactory.java:152)
…
-
https://beta.openai.com/tokenizer says:
> If you need a programmatic interface for tokenizing text, check out the [transformers](https://huggingface.co/docs/transformers/model_doc/gpt2#transformers…
-
I was looking with interest at your library and I was wondering if "applying LLMs on large documents" is on our feature list ?
I have done this ones ad hoc, so I mean:
- split the text in variou…
-
> If you're using Dokku - especially for commercial purposes - consider donating to project development via [OpenCollective](https://opencollective.com/dokku) or [Patreon](https://www.patreon.com/dokk…
-
# Crash report
### What happened?
I'm experiencing a persistent issue with a Tkinter-based GUI application on macOS. When running the script directly in the terminal, everything functions correctly,…