search

microsoft / unilm

Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities
https://aka.ms/GeneralAI
MIT License
19.09k stars 2.44k forks source link

1-bit LLM for embedding #1496

Open nachtsky1077 opened 2 months ago

nachtsky1077 commented 2 months ago

Hi, I am wondering if there's any work on leveraging the 1-bit LLM(BitNet) as embedding model? Thanks!