search

MinishLab / model2vec

Distill a Small Static Model from any Sentence Transformer
https://minishlab.github.io/
MIT License
413 stars 18 forks source link

Support for LLM2VEC models? #105

Closed sandeep-krutrim closed 2 weeks ago

sandeep-krutrim commented 3 weeks ago

Hi

Thanks for this great work !

It would be awesome if you can provide distillation support for LLM2VEC models. Sentence Transformer currently doesnt have a support for LLM2VEC and hence it is not possible to use it for distillation.

Thanks

stephantul commented 3 weeks ago

Hello @sandeep-krutrim, thanks for reaching out!

We don't use sentence-transformers to distill models, just plain transformers. We basically just do a forward pass, nothing fancy. I'm not familiar with the library itself, so I don't know what would be needed to properly support them.

If you have a clear example of how you do a forward pass, we could add support or tell you how it could work.

Stéphan

stephantul commented 2 weeks ago

Hello, I'm closing this for now because of inactivity.

Let me know if you need more help! Stéphan