knuddelsgmbh / jtokkit

JTokkit is a Java tokenizer library designed for use with OpenAI models.
https://jtokkit.knuddels.de/
MIT License
516 stars 38 forks source link

chore(deps): update dependency tiktoken to v0.5.1 #55

Closed renovate[bot] closed 9 months ago

renovate[bot] commented 9 months ago

Mend Renovate

This PR contains the following updates:

Package Change Age Adoption Passing Confidence
tiktoken (changelog) ==0.4.0 -> ==0.5.1 age adoption passing confidence

Release Notes

openai/tiktoken (tiktoken) ### [`v0.5.1`](https://togithub.com/openai/tiktoken/blob/HEAD/CHANGELOG.md#v051) [Compare Source](https://togithub.com/openai/tiktoken/compare/0.5.0...0.5.1) - Add `encoding_name_for_model`, undo some renames to variables that are implementation details ### [`v0.5.0`](https://togithub.com/openai/tiktoken/blob/HEAD/CHANGELOG.md#v050) [Compare Source](https://togithub.com/openai/tiktoken/compare/0.4.0...0.5.0) - Add `tiktoken._educational` submodule to better document how byte pair encoding works - Ensure `encoding_for_model` knows about several new models - Add `decode_with_offets` - Better error for failures with the plugin mechanism - Make more tests public - Update versions of dependencies

Configuration

📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

â™» Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.



This PR has been generated by Mend Renovate. View repository job log here.