search

brainless / dwata

AI assistants for everyone
https://dwata.com
Apache License 2.0
124 stars 7 forks source link

Integrate with rust-bert #111

Open brainless opened 2 months ago

brainless commented 2 months ago

As a user I would like to use local libraries to generate embedding. On their website rust-bert is introduced as:

Rust-native state-of-the-art Natural Language Processing models and pipelines. Port of Hugging Face’s Transformers library, using tch-rs or onnxruntime bindings and pre-processing from rust-tokenizers.

We can use this for classification, summarization or other utilities as well in the future.

brainless commented 2 months ago

It would be nice to have this done for our MVP, as part of the work for #105.