search

michaelfeil / hf-hub-ctranslate2

Connecting Transformers on HuggingFace Hub with CTranslate2
https://michaelfeil.github.io/hf-hub-ctranslate2/
MIT License
32 stars 2 forks source link

Don't understand the sentence_embedding implementation #18

Open jgcb00 opened 11 months ago

jgcb00 commented 11 months ago

Hi,

I don't get how the code is working, So the main idea is to replace the forward method of the model to use Ctranslate2 Model instead of the classic forward method of sentence transformers.

But in the case I want to get the sentence embeddings in sentence_transformers there is this line 181 : 

embeddings = out_features[output_value]

output_value = 'sentence_embedding' by default, so it must have been calculated in the forward method because everything before is in an if statement.

The forward method, is supercharged by this one, so I should get somewhere in the forward method : features["sentence_embedding"] as it is returning features, but it's just not there, so I don't get how you managed to make it works....

Thanks for your time

michaelfeil commented 9 months ago

I recommend you to check out: https://github.com/michaelfeil/infinity for better sentence transformer support!

michaelfeil commented 9 months ago

Otherwise, this might work: https://github.com/michaelfeil/hf-hub-ctranslate2/blob/main/hf_hub_ctranslate2/ct2_sentence_transformers.py