search

deepset-ai / FARM

:house_with_garden: Fast & easy transfer learning for NLP. Harvesting language models for the industry. Focus on Question Answering.
https://farm.deepset.ai
Apache License 2.0
1.74k stars 248 forks source link

full dpr evalution added #715

Closed tstadel closed 2 years ago

tstadel commented 3 years ago

dpr_evaluation.py script contains average rank evaluation over the full dpr dataset. In order to make this work some minor changes had to be made to TextSimilarityHead and BaseBiAdaptiveModel.

Timoeller commented 3 years ago

Hey @tstadel thanks for the contribution. A better DPR evaluation is really needed to steer the training process.

We would take your code and make some adjustments so we can use the evaluation in our Evaluator object. See #716 We will work on it in the coming sprint and will leave this PR open for now. Hope that is fine? Thanks again for this important feature.

tstadel commented 3 years ago

Sure, you're welcome!

stale[bot] commented 3 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 21 days if no further activity occurs.

stale[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 21 days if no further activity occurs.