search

pixlie / dwata

AI enabled insights from emails, calendars, contacts, files, Slack, databases, web... Fast, private and local. Launching soon!
https://pixlie.com/dwata
Apache License 2.0
148 stars 7 forks source link

Integrate with rust-bert #111

Open brainless opened 5 months ago

brainless commented 5 months ago

As a user I would like to use local libraries to generate embedding. On their website rust-bert is introduced as:

Rust-native state-of-the-art Natural Language Processing models and pipelines. Port of Hugging Face’s Transformers library, using tch-rs or onnxruntime bindings and pre-processing from rust-tokenizers.

We can use this for classification, summarization or other utilities as well in the future.

brainless commented 5 months ago

It would be nice to have this done for our MVP, as part of the work for #105.