How can I do Evaluation only by model and Evaluator

[mylovecc2020]

Thanks for your works！ Now I want to evaluation the model by test data, but I failed!

In examples i found this:

3) Create a sentence transformer model to glue both models together

model = SentenceTransformer(modules=[word_embedding_model, pooling_model])

sts_reader = STSBenchmarkDataReader('../datasets/stsbenchmark') evaluator = EmbeddingSimilarityEvaluator.from_input_examples(sts_reader.get_examples("sts-test.csv"))

model.evaluate(evaluator)

And I do like this: model = SentenceTransformer('bert-base-uncased') evaluater = CEBinaryClassificationEvaluator(sentence, scores) model.evaluate(evaluater)

there is an error: ----> 9 model.evaluate(evaluater)

2 frames /usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in getattr(self, name) 777 return modules[name] 778 raise ModuleAttributeError("'{}' object has no attribute '{}'".format( --> 779 type(self).name, name)) 780 781 def setattr(self, name: str, value: Union[Tensor, 'Module']) -> None:

ModuleAttributeError: 'SentenceTransformer' object has no attribute 'predict'

Does I use a wrong class? or other problems? thanks!

[nreimers]

Evaluators starting with CE are for CrossEncoders and don't work with SentenceTransformers BiEncoders.

See the documentation for the suitable evaluators for BiEncoders

[mylovecc2020]

thanks very much，I thought Biencoder and Crossencoder inherited from the same class，and have the same methods.

[dinuduke]

@nreimers Can you suggest me a blog/code snippet which can help me to reproduce the same performance results as Sbert has (screenshot attached for the same)

Also I have read Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks couple of times and I can see they have used Senteval for evaluation and I would like to reproduce the same(screenshot attached) for unsupervised document similarity problem. How can I implement this ?

The above annotated are probably old evaluation techniques and may not fit for unsupervised problem statement like( Comparing two blogs based on the semantic understanding not lexicon one). For this we have STS 12-16 and followed by others(screenshot attached).

Note :

1. I am going to used this information for study and research purpose. The basic idea is to reproduce all the results attached above.
2. Any Help with Code and implementation part will be helpful.
3. I will also try to share my findings with the community via blogs and other means.

Thanks in advance.

Dinesh Singh

[nreimers]

@dinuduke Not sure what your question is. You find all the code here in the repository. SentEval code can be found in the senteval repository