Add `new_with_tokenizer` for `SentenceEmbeddingsModel`

guillaume-be / rust-bert

Rust native ready-to-use NLP pipelines and transformer-based models (BERT, DistilBERT, GPT2,...)

https://docs.rs/crate/rust-bert

Apache License 2.0

2.67k stars 216 forks source link

Closed guillaume-be closed 1 year ago

guillaume-be commented 1 year ago

Addition of constructor allowing passing custom tokenizers for sentence embeddings pipelines