huggingface/tokenizers (tokenizers)
### [`v0.20.4`](https://redirect.github.com/huggingface/tokenizers/releases/tag/v0.20.4)
[Compare Source](https://redirect.github.com/huggingface/tokenizers/compare/v0.20.3...v0.20.4)
#### What's Changed
- More cache options. by [@Narsil](https://redirect.github.com/Narsil) in [https://github.com/huggingface/tokenizers/pull/1675](https://redirect.github.com/huggingface/tokenizers/pull/1675)
- Disable caching for long strings. by [@Narsil](https://redirect.github.com/Narsil) in [https://github.com/huggingface/tokenizers/pull/1676](https://redirect.github.com/huggingface/tokenizers/pull/1676)
- Testing ABI3 wheels to reduce number of wheels by [@Narsil](https://redirect.github.com/Narsil) in [https://github.com/huggingface/tokenizers/pull/1674](https://redirect.github.com/huggingface/tokenizers/pull/1674)
- Adding an API for decode streaming. by [@Narsil](https://redirect.github.com/Narsil) in [https://github.com/huggingface/tokenizers/pull/1677](https://redirect.github.com/huggingface/tokenizers/pull/1677)
- Decode stream python by [@Narsil](https://redirect.github.com/Narsil) in [https://github.com/huggingface/tokenizers/pull/1678](https://redirect.github.com/huggingface/tokenizers/pull/1678)
- Fix encode_batch and encode_batch_fast to accept ndarrays again by [@diliop](https://redirect.github.com/diliop) in [https://github.com/huggingface/tokenizers/pull/1679](https://redirect.github.com/huggingface/tokenizers/pull/1679)
**Full Changelog**: https://github.com/huggingface/tokenizers/compare/v0.20.3...v0.20.4-rc0
Configuration
📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
🔕 Ignore: Close this PR and you won't be reminded about this update again.
[ ] If you want to rebase/retry this PR, check this box
This PR contains the following updates:
0.20.3->0.20.4Release Notes
huggingface/tokenizers (tokenizers)
### [`v0.20.4`](https://redirect.github.com/huggingface/tokenizers/releases/tag/v0.20.4) [Compare Source](https://redirect.github.com/huggingface/tokenizers/compare/v0.20.3...v0.20.4) #### What's Changed - More cache options. by [@Narsil](https://redirect.github.com/Narsil) in [https://github.com/huggingface/tokenizers/pull/1675](https://redirect.github.com/huggingface/tokenizers/pull/1675) - Disable caching for long strings. by [@Narsil](https://redirect.github.com/Narsil) in [https://github.com/huggingface/tokenizers/pull/1676](https://redirect.github.com/huggingface/tokenizers/pull/1676) - Testing ABI3 wheels to reduce number of wheels by [@Narsil](https://redirect.github.com/Narsil) in [https://github.com/huggingface/tokenizers/pull/1674](https://redirect.github.com/huggingface/tokenizers/pull/1674) - Adding an API for decode streaming. by [@Narsil](https://redirect.github.com/Narsil) in [https://github.com/huggingface/tokenizers/pull/1677](https://redirect.github.com/huggingface/tokenizers/pull/1677) - Decode stream python by [@Narsil](https://redirect.github.com/Narsil) in [https://github.com/huggingface/tokenizers/pull/1678](https://redirect.github.com/huggingface/tokenizers/pull/1678) - Fix encode_batch and encode_batch_fast to accept ndarrays again by [@diliop](https://redirect.github.com/diliop) in [https://github.com/huggingface/tokenizers/pull/1679](https://redirect.github.com/huggingface/tokenizers/pull/1679) **Full Changelog**: https://github.com/huggingface/tokenizers/compare/v0.20.3...v0.20.4-rc0Configuration
📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
🔕 Ignore: Close this PR and you won't be reminded about this update again.
This PR was generated by Mend Renovate. View the repository job log.