Token Metrics Incorrect For Large Datasets - Githubissues

sambanova / generative_data_prep

Apache License 2.0

58 stars 8 forks source link

Token Metrics Incorrect For Large Datasets #53

Closed snova-zoltanc closed 9 months ago

snova-zoltanc commented 1 year ago

snova-zoltanc commented 9 months ago

Fixed in https://github.com/sambanova/generative_data_prep/pull/79