search

dqbd / tiktoken

JS port and JS/WASM bindings for openai/tiktoken
MIT License
649 stars 49 forks source link

Add new models #51

Closed rodumani closed 1 year ago

rodumani commented 1 year ago

Hi! I'm so glad that I can contribute to the awesome package.

I've added the following new models introduced on June 13, 2023.

zerfl commented 1 year ago

You'll want to update the wasm bindings as well to make this work.

See https://github.com/zerfl/tiktoken/commit/7ed984eed3ec2b7d15b35a2eeb09278cfd16af03#diff-3920f5b0c1e7a80683f05ae43cb02274ad7d418643021a88b4cf765008c79779

dqbd commented 1 year ago

Looks fine to me! Thanks @rodumani @zerfl!

tractorcow commented 1 year ago

The new models are missing from typescript definition. See 

export type TiktokenModel =

in wasm/src/lib.rs