tingofurro / summac

Codebase, data and models for the SummaC paper in TACL
https://arxiv.org/abs/2111.09525
Apache License 2.0
81 stars 20 forks source link

Threshold tuning #5

Open m0baxter opened 1 year ago

m0baxter commented 1 year ago

I was looking at your code and attempting to recreate your results.

If this this is how the results quoted in the paper were obtained it seems a bit strange that you are fine-tuning your threshold on the test set. Not withstanding the fact that the threshold is tuned per dataset in the benchmark (this being mentioned in the paper).