Bump ai.djl.huggingface:tokenizers from 0.18.0 to 0.21.0

[dependabot[bot]]

Bumps ai.djl.huggingface:tokenizers from 0.18.0 to 0.21.0.

Release notes

Sourced from ai.djl.huggingface:tokenizers's releases.

DJL v0.20.0 release

Key Features

• Upgrades and enhancements for Engines
  • Upgrades PyTorch to 1.13.0 (#2157)
  • Add support for Apple's Metal Performance Shaders (MPS) in PyTorch (#2037)
  • Add system property to config GraphExecutorOptimize (#2156)
  • Upgrades ONNXRuntime to 1.13.1 (#2115)
  • Upgrades Paddle to 2.3.2 (#2116)
  • Upgrades TensorFlow to 2.7.4 (#2121)
  • Upgrades HuggingFace tokenizer version to 0.13.1 (#2127)
  • Upgrades XGBoost to 1.7.1 (#2143)
• DJLServing
  • Adds large model inference support with MPI mode (#291)
  • Adds built-in DeepSpeed handler (#292)
  • Publishes PaddlePaddle docker image (#342)
• Adds TabNet Training (#2057)
• Publishes DJL Zero (#2091)
• Adds Spark extension (#2162)
• Introduces several improvements for timeseries extension
• Adds ImageFeatureExtractor example and resnet base model to model zoo

Enhancement

• Introduces several enhancements for timeseries extension:
  • Adds probability distribution support for timeseries (#2025)
  • Add time series dataset support for timeseries package (#2026)
  • Update M5Forecast dataset and its unittest (#2105)
  • Add some basic block and deepAR model (#2027)
  • Enable pytorch deepar model inference in time series package (#2149)
• Introduces several enhancement for HuggingFace tokenizer:
  • Enhance huggingface text embedding translator to support max length padding (#2049)
  • Add cli options to only validate jit model on CPU (#2052)
  • Add batch decoding methods for tokenizers (#2154)
• Adds new models to DJL model zoo:
  • Adds TabNet model for tabular dataset in modelzoo (#2036)
  • Adds yolo5s to OnnxRuntime model zoo (#2046)
  • Object Detection (#1930)
  • Adds image classification resnet18 base model to model zoo (#2079)
• DJL API improvements:
  • Adds Sparsemax block (#2028)
  • Updates the SemanticSegmentationTranslator (#2032)
  • Creates Ensembleable (#2043)
  • Handle error when forget to initialize a child block (#2045)
  • Adds draw mask for BitMapWrapper (#2071)
  • Allows show NDArray content in debugger (#2078)
  • Rename transparency to opacity in CategoryMask (#2081)
  • Allows to show NDArray content in Debugger 2 (#2080)
  • Transfer learning with pytorch engine on fresh fruit dataset (#2070)
  • Ensure GradientCollector can clear gradients (#2101)

... (truncated)

Commits

• 1a5d3c5 Remove -SNAPSHOT for release v0.21.0
• 5683e57 Bump up DJL version to 0.21.0
• 839be67 [spark] Add Numpy binary translator (#2399)
• 50d5d6a [sentencepiece] Adds centos 7 support (#2402)
• 4fcecdd Removes DLR github actions (#2398)
• e29a563 [api] Improve Criteria.build() error message (#2397)
• e53b161 [spark] Integrate HuggingFace tokenizer (#2311)
• 039f11e [docs] Improve documents (#2396)
• 04cf346 Remove performance issues from freezing MXNet (#2394)
• 7ef827a update onnxruntime version to 1.14.0 (#2393)
• Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

---

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)