search

meta-llama / llama

Inference code for Llama models
Other
56.1k stars 9.53k forks source link

Sentence/ Word embedding from LLaMA #152

Open kmukeshreddy opened 1 year ago

kmukeshreddy commented 1 year ago

Hello,

Could you please let me know if there is a provision to get sentence embeddings from LLaMA? If yes, could you please the sample reference code?

Could you please let me know whether Zero-shot classification is available in LLaMA? If yes, could you please share the reference?

sstolpovskiy commented 1 year ago

Looks like LLama embedding available using HF https://huggingface.co/shalomma/llama-7b-embeddings

wb14123 commented 1 year ago

@sstolpovskiy I don't see any mentioning of embedding in the project's readme. Do you have more information about how to use the model?

kmukeshreddy commented 1 year ago

@sstolpovskiy, Thank you for the link at Huggingface. I was looking for an official API similar to openai. Please let me know if you have tried the HF LLAma Api. How does the performance look?

prasad4fun commented 1 year ago

Hi Any update on this link, could you use Llama instead of openAI for sentence embeddings?

hjerbii commented 1 year ago

Hello, any updates on how to generate embeddings using Llama? Thanks!

kmukeshreddy commented 1 year ago

No update; I'm waiting for a reply.

realliyifei commented 1 year ago

How to get the sentence embedding of llama-2? (I believe the previous post is about llama-1)

sujantkumarkv commented 1 year ago

@realliyifei We can get llama-2 embeddings with llama.cpp repo as show in this subreddit, here

after we build, we get an embedding file which we can run locally, its fast enough but i'm not sure how this would scale for say million tokens or so.

I'm looking for elegant/simple solutions for it too, maybe offload to a gpu at last & we can mostly store embeddings locally (in most cases we dont need a vector database many industry titans including karpathy sir advocate this) let me know your updates.