Fix encoding_for_model to return nil for unknown encoding

IAPark / tiktoken_ruby

Unofficial ruby binding for tiktoken by way of rust

MIT License

109 stars 26 forks source link

Fix encoding_for_model to return nil for unknown encoding #29

Closed rob-mindtrip closed 3 months ago

rob-mindtrip commented 4 months ago

This pull request addresses issue #28.

Currently, when you pass an unrecognized model name to Tiktoken.encoding_for_model, instead of returning nil it returns an unexpected Hash, making it more difficult to detect whether the encoding is available. This seems to have been introduced by 0c1a45b46a

Added a simple spec to confirm.