Bump tiktoken from 0.4.0 to 0.6.0

[dependabot[bot]]

Bumps tiktoken from 0.4.0 to 0.6.0.

Changelog

Sourced from tiktoken's changelog.

[v0.6.0]

• Optimise regular expressions for a 20% performance improvement, thanks to @​paplorinc!
• Add text-embedding-3-* models to encoding_for_model
• Check content hash for downloaded files
• Allow pickling Encoding objects. Registered Encoding will be pickled by reference
• Workaround PyO3 bug for frozenset conversion

Thank you to @​paplorinc, @​mdwelsh, @​Praneet460!

[v0.5.2]

• Build wheels for Python 3.12
• Update version of PyO3 to allow multiple imports
• Avoid permission errors when using default cache logic

[v0.5.1]

• Add encoding_name_for_model, undo some renames to variables that are implementation details

[v0.5.0]

• Add tiktoken._educational submodule to better document how byte pair encoding works
• Ensure encoding_for_model knows about several new models
• Add decode_with_offets
• Better error for failures with the plugin mechanism
• Make more tests public
• Update versions of dependencies

Commits

• 84d88dc Allow use of gpt-2 and gpt-3.5 in encoding_for_model (#185)
• 01df436 Update cibuildwheel
• 89153d7 Sync codebase
• 55c8d83 added two new embedding model's encoding (#247)
• 6cc3a46 Optimize regular expressions used for splitting by ~20% (#234)
• db5bda9 Clarify language models in README (#203)
• 3ee6c35 Add support for checking hash of downloaded files before use. (#230)
• 9e79899 Sync codebase
• 6267f91 Sync codebase
• 39f29ce Sync codebase
• Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

---

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)