issues
search
openai
/
tiktoken
tiktoken is a fast BPE tokeniser for use with OpenAI's models.
MIT License
11.61k
stars
785
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
ValueError: Unknown encoding cl100k_base error when program is compiled
#168
tomhayw
closed
1 year ago
1
✨ feat: Split code rust, for run as python lib and rust lib
#167
Miuler
opened
1 year ago
5
Deprecation of older models
#166
AlexanderMorou
closed
1 year ago
3
Token
#165
leotorenzo6
closed
1 year ago
0
Pricing
#164
macarneiro1
closed
1 year ago
1
Unknown encoding cl100k_base when building program using pyinstaller
#163
Klyser8
closed
1 year ago
1
BOOL-1940: Refactor code to work with Python 3.7
#162
Mainstay-Noah-Huppert
closed
1 year ago
0
Fatal Python error: Segmentation fault from `tiktoken/core.py`
#161
yuvalshi0
closed
1 year ago
3
Calculate token consumption for OpenAI function definitions
#160
Pwuts
closed
1 year ago
1
Inconsistent "\n\n" encoding
#159
vince62s
closed
1 year ago
4
大量tiktok真人实号
#158
Tbagbast
closed
1 year ago
0
Feature Request: return dollar amount for OpenAI request response
#157
dhruv-anand-aintech
closed
1 year ago
1
Do you have a Java version of tiktoken
#156
gitcyh
closed
1 year ago
1
ValueError: Unknown encoding text-embedding-ada-002
#155
heavenkiller2018
closed
1 year ago
1
What is the env RAYON_NUM_THREADS?
#154
chzhyang
closed
1 year ago
0
How to get available requests
#153
devGetWorld
closed
1 year ago
1
fix pycache folder being added instead of encoding info files
#152
dheerajiiitv
opened
1 year ago
9
mypy: Cannot find implementation or library stub for module named "tiktoken" [import]
#151
ryan-mccaffrey
closed
1 year ago
1
I'm consistently getting this `wasm RuntimeError: unreachable` for large files
#150
manikanta-kondeti
closed
1 year ago
1
An unusual loss pattern when training a model from scratch
#149
ericxsun
closed
1 year ago
1
Add new types to TiktokenModel
#148
phil-willowtree
closed
1 year ago
3
Cannot find the educational submodule in tiktoken
#147
bkowshik
closed
11 months ago
5
update gpt-16k
#146
hyh1750522171
closed
1 year ago
0
SSL: Certificate_Verify_Failed
#145
Kwalladee
closed
1 year ago
1
better visualization for the merge process in the educational implementation
#144
youkaichao
closed
11 months ago
10
from __future__ import annotations to use dict for type annotation
#143
youkaichao
closed
1 year ago
0
ValueError: Unknown encoding cl100k_base
#142
nnnnnb
closed
1 year ago
1
ImportError: PyO3 modules may only be initialized once per interpreter process
#141
jackyhzzj
closed
8 months ago
13
Cost for failed GPT requests
#140
krishna-cactus
closed
1 year ago
1
Usage with Claude's BPE vocab
#139
19h
closed
1 year ago
1
Cannot import tiktoken into flask app - PyO3 modules may only be initialized once per interpreter process
#138
simondpalmer
closed
1 year ago
1
Any introduction about api `encode_with_unstable`?
#137
fseasy
opened
1 year ago
2
ValueError: not enough values to unpack (expected 2, got 1)->for token, rank in (line.split() for line in contents.splitlines() if line)
#136
pandaGost
closed
1 year ago
3
Add support for fine-tuned models in encoding_for_model
#135
thespino
opened
1 year ago
2
wasm version or pure python version needed
#134
mccoysc
closed
1 year ago
2
ImportError: cannot import name '_tiktoken' from partially initialized module 'tiktoken'
#133
onlychenchen
closed
1 year ago
2
Maybe insight into closed issue 46?
#132
TradingPlacesResearch
closed
1 year ago
2
"pip install tiktoken" fails when I execute "docker build" for PyPy and amd64 on an M1 Mac.
#131
aikawa-7
closed
1 year ago
2
-
#130
jsnijdewint
closed
1 year ago
0
The walrus operator prevents the package from working for Python 3.7.
#129
LuisGarciaGomez
closed
1 year ago
1
Option to use env variable for BPE_FILE location
#128
csnelsonchu
opened
1 year ago
0
how to import sentencepiece bpe vocabulary into tiktoken?
#127
qiancheng99
closed
1 year ago
1
Cannot import tiktoken need Pyo3 migration
#126
simondpalmer
closed
1 year ago
7
update license metadata to use SPDX identifier as per recommendation
#125
imrehg
opened
1 year ago
0
Not able to install tiktoken from pypi
#124
goharyounas
closed
1 year ago
1
Eagerly anticipating GPT-4 32k model support.
#123
harrison001
closed
1 year ago
2
fix typo in error message (tiktok -> tiktoken)
#122
tmyjoe
closed
1 year ago
2
Does anyone take development tasks? I need someone to help me develop and deploy an OPENAI API reverse proxy with statistical functions.
#121
SunnyGPT
closed
1 year ago
2
The calculation results different with Tokenizer
#120
mysaber
closed
1 year ago
4
Get_encoding failures in aws lambda execution
#119
jhb96
closed
1 year ago
1
Previous
Next