search

hxy9243 / ragtoy

This is a toy Python project to learn and explore the capabilities of openAI chatGPT API.
MIT License
0 stars 0 forks source link

Bump tiktoken from 0.3.3 to 0.8.0 #58

Open dependabot[bot] opened 1 month ago

dependabot[bot] commented 1 month ago

Bumps tiktoken from 0.3.3 to 0.8.0.

Changelog

Sourced from tiktoken's changelog.

[v0.8.0]

  • Support for o1- and chatgpt-4o- models
  • Build wheels for Python 3.13
  • Add possessive quantifiers to limit backtracking in regular expressions, thanks to @​l0rinc!
  • Provide a better error message and type for invalid token decode
  • Permit tuples in type hints
  • Better error message for passing invalid input to get_encoding
  • Better error messages during plugin loading
  • Add a __version__ attribute
  • Update versions of pyo3, regex, fancy-regex
  • Drop support for Python 3.8

[v0.7.0]

  • Support for gpt-4o
  • Performance improvements

[v0.6.0]

  • Optimise regular expressions for a 20% performance improvement, thanks to @​paplorinc!
  • Add text-embedding-3-* models to encoding_for_model
  • Check content hash for downloaded files
  • Allow pickling Encoding objects. Registered Encoding will be pickled by reference
  • Workaround PyO3 bug for frozenset conversion

Thank you to @​paplorinc, @​mdwelsh, @​Praneet460!

[v0.5.2]

  • Build wheels for Python 3.12
  • Update version of PyO3 to allow multiple imports
  • Avoid permission errors when using default cache logic

[v0.5.1]

  • Add encoding_name_for_model, undo some renames to variables that are implementation details

[v0.5.0]

  • Add tiktoken._educational submodule to better document how byte pair encoding works
  • Ensure encoding_for_model knows about several new models
  • Add decode_with_offets
  • Better error for failures with the plugin mechanism
  • Make more tests public
  • Update versions of dependencies

[v0.4.0]

  • Add decode_batch and decode_bytes_batch

... (truncated)

Commits
  • 6352764 Alas, emulation
  • ed27aa3 Attempt to fix emulated builds
  • 05e66e8 Partial sync of codebase
  • 9f7f69d Add possessive quantifiers to avoid catastrophic backtracking (#258)
  • c0ba74c update README to mention gpt-4o
  • bfe00ad Bump cibuildwheel
  • 9d01e56 Sync codebase
  • 1b9faf2 Simplify byte_pair_merge (#255)
  • 6defed5 Inline custom mapping function in _byte_pair_merge
  • b4c687e Avoid calling byte_pair_encode for existing tokens
  • Additional commits viewable in compare view


Dependabot compatibility score

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)