search

tuetschek / e2e-metrics

E2E NLG Challenge Evaluation metrics
http://www.macs.hw.ac.uk/InteractionLab/E2E/
Other
91 stars 22 forks source link

using TorchMetrics #2

Open Borda opened 2 years ago

Borda commented 2 years ago

Hello, I am curious if you would be interested in using TorchMetrics which has recently implemented various Text-related metrics? One of your students, @stancld have helped us with adding these metrics, so he may also advise of any potential integration... :)

Thank you for your consideration :rabbit:

tuetschek commented 2 years ago

Hi @Borda , thanks for the pointer and apologies for a rather negative reply. I haven't been developing this repo any further for the past ~2 years, so I'm not planning to include any new metrics here.

You might try asking at (or get involved in) https://github.com/GEM-benchmark/GEM-metrics, which is basically an evolution of this. (Note that there probably would need to be a metric in TorchMetrics, not yet included in GEM, for the project to consider using it.)