search

megagonlabs / sato

Code and data for Sato https://arxiv.org/abs/1911.06311.
Apache License 2.0
108 stars 40 forks source link

About the validation F1 score #23

Open Lpn-98 opened 3 years ago

Lpn-98 commented 3 years ago

Hi, I am a student who is a novice in the research aspect of NLP, I runned the code as README。 Before, I got warning about Early Stopping and UndefinedMetricWarning, now I know what they are, but now I came up with another problem:

In the file .\notebooks\plot_data, the F1 scores of the result are greatly higher than the Table 1 of the paper. For example, for "result_CRF_LDA_multi-col", the macro F1 (which is =AVERAGE(E2:E77)) is 0.90, and the support weighted F1 (which is =SUMPRODUCT(E2:E77,F2:F77)/SUM(F2:F77)) is 0.97, while in the Table 1 of the paper they are around 0.735 and 0.925.

I have no idea why I get this problem, has anyone can give me a hint? Thank you!

Lpn-98 commented 3 years ago

And I found that some files in the 'outputs' have userid as filename extension, like:

'/home/dalab/Desktop/do_not_delete-lpn_code/sato-master/results/CRF_log/type78/CRF+LDA_pathL_multi-col/marco avg-val/f1-score/events.out.tfevents.1632806349.dalab'

This file has filename extension '.delab', which is the userid of my ubuntu. I donot know whow to open this file and get the F1 score. Does anyone have and ideas? Thanks very much.