feat: faster rope, nomic embed v1.5 binary and matryoshka - Githubissues

taylorai / mlx_embedding_models

run embeddings in MLX

MIT License

62 stars 6 forks source link

feat: faster rope, nomic embed v1.5 binary and matryoshka #5

Closed zanussbaum closed 2 months ago

zanussbaum commented 2 months ago

rope for mlx expects shape (B, H, S , D) instead of (B, S, H, D)
nomic embed v1.5 uses a non-affine layernorm for binary embeddings
adds support for matryoshka dimensions for nomic embed v1.5