issues
search
tryAGI
/
Tiktoken
This project implements token calculation for OpenAI's gpt-4 and gpt-3.5-turbo model, specifically using `cl100k_base` encoding.
https://github.com/openai/tiktoken
MIT License
48
stars
2
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
build(deps): Bump Verify.MSTest from 26.2.0 to 26.3.1 in the all group
#51
dependabot[bot]
closed
4 days ago
1
build(deps): Bump Microsoft.NET.Test.Sdk from 17.10.0 to 17.11.0 in the all group
#50
dependabot[bot]
closed
1 week ago
1
build(deps): Bump the all group with 2 updates
#49
dependabot[bot]
closed
2 weeks ago
0
Improve memory usage of `ModelToEncoder`
#48
mikethea1
opened
3 weeks ago
0
build(deps): Bump the all group with 4 updates
#47
dependabot[bot]
closed
3 weeks ago
0
build(deps): Bump Verify.MSTest from 26.1.5 to 26.1.6 in the all group
#46
dependabot[bot]
closed
1 month ago
0
Suggestion regarding the style of API
#45
ddobric
opened
1 month ago
3
build(deps): Bump Verify.MSTest from 26.0.0 to 26.1.5 in the all group
#44
dependabot[bot]
closed
1 month ago
0
build(deps): Bump the all group with 3 updates
#43
dependabot[bot]
closed
1 month ago
0
build(deps): Bump Verify.MSTest from 25.3.0 to 25.3.1 in the all group
#42
dependabot[bot]
closed
1 month ago
0
feat: Initial tokenizer.json support
#41
HavenDV
opened
1 month ago
0
Generate/load Encoder from tokenizer.json file
#40
michalblaha
opened
1 month ago
1
build(deps): Bump Verify.MSTest from 25.0.4 to 25.3.0 in the all group
#39
dependabot[bot]
closed
2 months ago
0
build(deps): Bump DotNet.ReproducibleBuilds from 1.1.1 to 1.2.4 in the all group
#38
dependabot[bot]
closed
2 months ago
0
build(deps): Bump Verify.MSTest from 25.0.3 to 25.0.4 in the all group
#37
dependabot[bot]
closed
2 months ago
0
build(deps): Bump Verify.MSTest from 25.0.1 to 25.0.3 in the all group
#36
dependabot[bot]
closed
2 months ago
0
Throwing OutOfMemoryException while getting encoding model
#35
CodePlacer
opened
3 months ago
0
build(deps): Bump the all group with 4 updates
#34
dependabot[bot]
closed
3 months ago
0
build(deps): Bump the all group with 3 updates
#33
dependabot[bot]
closed
3 months ago
0
build(deps): Bump the all group with 2 updates
#32
dependabot[bot]
closed
3 months ago
0
Token counting for a whole message
#31
Xan-Kun
opened
3 months ago
3
using Encoder ForModel(string modelName)
#30
Xan-Kun
closed
3 months ago
2
build(deps): Bump Verify.MSTest from 24.1.0 to 24.2.0 in the all group
#29
dependabot[bot]
closed
3 months ago
0
build(deps): Bump Verify.MSTest from 23.7.1 to 24.1.0 in the all group
#27
dependabot[bot]
closed
4 months ago
0
build(deps): Bump the all group with 5 updates
#26
dependabot[bot]
closed
5 months ago
0
build(deps): Bump the all group with 2 updates
#25
dependabot[bot]
closed
5 months ago
0
build(deps): Bump the all group with 1 update
#24
dependabot[bot]
closed
5 months ago
0
build(deps): Bump the all group with 1 update
#23
dependabot[bot]
closed
5 months ago
0
build(deps): Bump the all group with 4 updates
#22
dependabot[bot]
closed
5 months ago
0
build(deps): Bump the all group with 4 updates
#21
dependabot[bot]
closed
5 months ago
1
build(deps): Bump the all group with 4 updates
#20
dependabot[bot]
closed
6 months ago
1
Add UTF-safe explore
#19
lofcz
closed
6 months ago
2
build(deps): Bump the all group with 10 updates
#18
dependabot[bot]
closed
6 months ago
0
build(deps): Bump the all group with 8 updates
#17
dependabot[bot]
closed
7 months ago
0
build(deps): Bump the all group with 9 updates
#16
dependabot[bot]
closed
7 months ago
0
build(deps): Bump the all group with 1 update
#15
dependabot[bot]
closed
9 months ago
0
build(deps): Bump the all group with 1 update
#14
dependabot[bot]
closed
9 months ago
0
build(deps): Bump the all group with 2 updates
#13
dependabot[bot]
closed
9 months ago
0
build(deps): Bump the all group with 1 update
#12
dependabot[bot]
closed
10 months ago
0
build(deps): Bump the all group with 2 updates
#11
dependabot[bot]
closed
10 months ago
0
build(deps): Bump the all group with 2 updates
#10
dependabot[bot]
closed
11 months ago
0
build(deps): Bump the all group with 1 update
#9
dependabot[bot]
closed
11 months ago
0
build(deps): Bump the all group with 1 update
#8
dependabot[bot]
closed
11 months ago
0
build(deps): Bump the all group with 2 updates
#7
dependabot[bot]
closed
11 months ago
0
build(deps): Bump the all group with 3 updates
#6
dependabot[bot]
closed
12 months ago
0
build(deps): Bump the all group with 1 update
#5
dependabot[bot]
closed
1 year ago
0
build(deps): Bump the all group with 3 updates
#4
dependabot[bot]
closed
1 year ago
0
Feature Request: Method to extract tokens as a list of strings
#2
cilerler
closed
1 year ago
5
Questions about cache usage
#1
JadynWong
closed
1 year ago
2