ankane / tokenizers-ruby

Fast state-of-the-art tokenizers for Ruby
Apache License 2.0
132 stars 6 forks source link

Add trainers #23

Closed petergoldstein closed 1 year ago

petergoldstein commented 1 year ago

The Ruby structure here is a little more complex to capture the default arguments in the Python API.