search

ankane / tokenizers-ruby

Fast state-of-the-art tokenizers for Ruby
Apache License 2.0
132 stars 6 forks source link

Fill out the set of getters and setters #24

Closed petergoldstein closed 1 year ago

petergoldstein commented 1 year ago

@ankane So I think that now this is merged, the gem is essentially feature complete with the Python library. Are there any major gaps you're aware of?

ankane commented 1 year ago

Nice, I think that's most of the API. Going to try and get the Sequence classes working later today and make sure all these examples run: https://huggingface.co/docs/tokenizers/pipeline

ankane commented 1 year ago

New release is out. Thanks again for all the great PRs!

petergoldstein commented 1 year ago

Great. My pleasure.