search

ankane / tokenizers-ruby

Fast state-of-the-art tokenizers for Ruby
Apache License 2.0
132 stars 6 forks source link

Add missing parameters to BpeTrainer #20

Closed petergoldstein closed 1 year ago

petergoldstein commented 1 year ago

Adds a number of missing parameters to the BpeTrainer