search

MinishLab / model2vec

Distill a Small Static Model from any Sentence Transformer
https://minishlab.github.io/
MIT License
417 stars 18 forks source link

fix: Add explicit errors for BPE and unigram, return tokenizer without cha… #54

Closed stephantul closed 1 month ago

stephantul commented 1 month ago

…nges if tokens_to_remove is empty

codecov[bot] commented 1 month ago

Codecov Report

Attention: Patch coverage is 56.09756% with 18 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
model2vec/distill/tokenizer.py 54.05% 17 Missing :warning:
model2vec/distill/distillation.py 75.00% 1 Missing :warning:
Files with missing lines Coverage Δ
model2vec/distill/distillation.py 96.29% <75.00%> (-1.14%) :arrow_down:
model2vec/distill/tokenizer.py 71.18% <54.05%> (-25.88%) :arrow_down: