yethee / tiktoken-php

This is a port of the tiktoken

MIT License

92 stars 19 forks source link

Add batch encoding capabilities #9

Closed junaidbinfarooq closed 6 months ago

junaidbinfarooq commented 6 months ago

Change proposed:

Adds support to encode text in batches
Test code refactor

yethee commented 6 months ago

Thanks for your contribution!