search

facebookresearch / online_dialog_eval

Code for the paper "Learning an Unreferenced Metric for Online Dialogue Evaluation", ACL 2020
Other
28 stars 7 forks source link

Human correlation notebook has errors #9

Open jsedoc opened 3 years ago

jsedoc commented 3 years ago

I attempted to run the notebook and it mostly works, but I had issues with some code blocks. Please see your notebook with some small deltas on Google Colab (https://colab.research.google.com/drive/10Ro6K14cpgTQuEV6aZKIU7Kzy2MFHsu-?usp=sharing). I've added try except blocks around the cells that are problems.

For the most part, I don't think the errors are a big deal, but I'd like to know what score_scaled should be. It is used here

conv_dial_scores.enjoy.corr(conv_dial_scores.score_scaled, method='pearson')

Thanks!

koustuvsinha commented 3 years ago

I think score_scaled was a leftover value cell from the previous notebook which I forked from. I checked my own commits from my private repo, it doesn't seem I used those three cells anywhere, and I just might have overlooked to clean them up. (This is why I hate working with notebooks due to their non-linear structure haha) I will clean those cells, thanks for pointing it out!