Bump tokenizers from 0.18.0 to 0.19.0

[dependabot[bot]]

Bumps tokenizers from 0.18.0 to 0.19.0.

Release notes

Sourced from tokenizers's releases.

DJL v0.19.0 release

Key Features

• Creates new LightGBM engine (#1895)
• Upgrades and enhancements for Engines
  • Upgrades PyTorch to 1.12.1 (#1894)
  • Upgrades ONNXRuntime to 1.12.1 (#1879)
  • Upgrades Apache MXNet to 1.9.1 (#1898)
  • Publishes new xgboost-gpu package to maven (#1918)
  • Adds ARM support for ONNXRuntime (#1856)
  • Disable autograd by default when PyTorch engine start (#1872)
• Introduces several enhancement for HuggingFace tokenizer
  • Introduces HuggingFace model zoo (#1984)
  • Adds a few built-in Translators for HuggingFace NLP models
  • Adds macOS M1 support for HuggingFace tokenizer
  • Adds ARM support for HuggingFace tokenizer
  • Adds centos 7 support for HuggingFace tokenizer (#1874)
  • Adds decode API for HuggingFace tokenizer (#1843)
  • Adds padding and truncation support for HuggingFace tokenizer (#1870)
  • Support stride in tokenizers (#2006)
• Introduces time series extension (#1903)
• Adds new Audio API and improves audio extension (#1974)
• Adds Android support for ONNXRuntime (#1844)
• JDK18 support (#1892)
• Adds python script to import HuggingFace model into DJL model zoo (#1835)
• DJLServing
  • Adds management console plugin, which allows user manage models with web UI (#205)
  • Adds KServe plugin (#177)
  • Publishes DeepSpeed docker image to dockerhub (#223)

Enhancement

• Adds a few more built-in Translators:
  • Adds HuggingFace QuestionAnsweringTranslator (#1828)
  • Adds HuggingFace FillMaskTranslator (#1876)
  • Adds HuggingFace TokenClassificationTranslator (#1906)
  • Adds HuggingFace TextClassificationTranslator (#1983)
  • Adds HuggingFace TextEmbeddingTranslator (#1953)
  • Adds speech recognition translator (#1899)
• Adds new models to DJL model zoo:
  • Adds PyTorch deeplabvs model into DJL model zoo (#1818)
  • Adds MobileNetV1 into model zoo (#1817)
• Image handling enhancement:
  • Improves ImageFactory to allow convert float32 NDArray to Image. (#1814)
  • Handle both HWC and CHW image (#1833)
• DJL API improvements:
  • Adds NDArray normalize() operator (#1924)
  • Adds DeferredTranslatorFactory to let serving.properties take effect (#1868)
  • Makes PtBertQATranslator compatible with huggingface model (#1827)
  • Improves debug log for model loading options. (#1825)

... (truncated)

Commits

• 37ef02d Remove -SNAPSHOT for release v0.19.0
• 9931df7 Increase DJL version to 0.19.0
• 5073966 fix some bugs in pytorch based examples (#2009)
• 93ec732 Creates a SystemNDManager interface (#1888)
• f54ef2e [example] add an example of time sereis model inferrence (#1971)
• a82b338 [fix] DynamicBuffer position error when expand for the firtst time (#2007)
• c974939 support stride in tokenizers (#2006)
• 4fa0ccf Upgrade dependency version and update README (#2005)
• 5ca256a [timeseries] add all frequencies support. (#1973)
• c4f162f Update LSTM.java (#2000)
• Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

---

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

[dependabot[bot]]

Superseded by #164.