search

tuetschek / ratpred

Referenceless quality estimation for natural language generation – predicting human ratings
Other
8 stars 0 forks source link

Improper masking on ranking instances #13

Closed tuetschek closed 5 years ago

tuetschek commented 5 years ago

Ranking instances are set to predict 1 for rating -- mask is by isnan not the presence of ranking!

tuetschek commented 5 years ago

Actually the mask is correct but the model still seems to use it in training somehow

tuetschek commented 5 years ago

Seems to be alright. Possibly fixed in 9a2bbf7b1d1cff7ad93d817838d14b3157b3f842.