Open ThiloteE opened 1 month ago
awesome! will take a look into your options!
For now, I got to install all-minilm-l6-v2
successfully. It's a local model.
Also JabRef distribution size is nearly unchanged (it's not a double)
@ThiloteE I think it's your time to shine! It seems Deep Java Library (djl) can run many many models with different architectures.
Maybe you can pick up some models from MTEB?
It seems you need to provide djl some kind of a URL of a model, and then it will magically handle it. Like this: djl://ai.djl.huggingface.pytorch/sentence-transformers/all-MiniLM-L6-v2
. Though, I have no idea (and googling didn't help) how to find those URLs. I opened a question on djl repo: https://github.com/deepjavalibrary/djl/discussions/3357
I think this documentation would be relevant for your question: https://docs.djl.ai/docs/load_model.html
Thanks for investigating @ThiloteE !
Models available in Model Zoo of Deep Java Library (meaning they are stored. probably, on their servers and it's easy to load them):
If the model is not in this list, we still can download it manually and run. Though, I currently investigating how to do this
@ThiloteE what do you think of https://huggingface.co/sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2 ?
Seems like a continuation of what we are currently using, but for many languages
You have noticed, that this model is from 2021, right? There must be better. Maybe, if push comes to shove and there is no other model in that size. Edit: I have added the model to the list.
How to add an embedding model to the Deep Java Library model zoo: https://djl.ai/docs/development/add_model_to_model-zoo.html
Steps to close this issue:
paraphrase-multilingual-MiniLM-L12-v2
into JabRef. --> @Ruslan paraphrase-multilingual-MiniLM-L12-v2
- and test w/ Chocolate.bib - and use the tests at https://blog.jabref.org/2024/08/21/AI-chatting/#ai-chat. - @ThiloteE paraphrase-multilingual-MiniLM-L12-v2
default in JabRef. - @InAnYan Follow-up: Write down documentation how to bring in a new model into DJL (if the link at https://github.com/InAnYan/jabref/issues/85#issuecomment-2329160668 is not enough - and updating the dependency).
This is a "living issue". Editing is appreciated.
Context:
Conditions:
Models:
ADR for (1) remote API vs (2) within JabRef vs (3) Separate Application
To be done.
(1) Advantage: Easy to implement; Currently probably better for multilingual (might change in future); Indexing can be faster than local, because computing can be offloaded to powerful remote hardware. Disadvantage: Costly. Conflicts with Open Source Ethos. Privacy converns. Rate limitations. Censorship. Dependency on external service.
(2) Advantage: JabRef does not depend on external application. Full freedom. No cost (apart from electricity / time). Close integration with JabRef's database. No privacy concerns. Disadvantage: Requires maintenance of JabRef. Increases size of JabRef. Can be slow, because code might not be optimized (e.g. offloading to GPU).
(3) Advantage: Separate applications can be highly optimized for AI workloads. More features. Better speed. More models. Are maintained. Almost all of them support and are compatible with the OpenAI API standard. Disadvantage: A separate application can require creating or maintenance of java bindings or plugin or (OpenAI) API compatibility. Requires installing and running a separate application. Needs maintenance of documentation for the separate application.
Implemenation roadmap:
[ ] Remote via API (Almost all embedding model providers support APIs that resemble OpenAI API)
[ ] "local" within JabRef
[ ] separate application