evanmiltenburg / Shared-task-on-NLG-Evaluation

Repository to organize a shared task on NLG evaluation
6 stars 0 forks source link

"Build it, break it"-style track? #2

Open evanmiltenburg opened 4 years ago

evanmiltenburg commented 4 years ago

Maybe it would be nice if people could also send in outputs they systematically manipulated themselves (introducing specific kinds of errors). Then we can see if the evaluation metrics proposed by others can actually pick up on those manipulations.