search

NJUNLP / GTS

Code and data for paper "Grid Tagging Scheme for Aspect-oriented Fine-grained Opinion Extraction". Aspect opinion pair datasets and aspect triplet datasets.
Apache License 2.0
85 stars 26 forks source link

Performance with five repeats #12

Open BD-MF opened 3 years ago

BD-MF commented 3 years ago

The experimental result in your article is the average of 5 repeats, but there is no trace of 5 repeats in the code. I tried your model initially, and I found that every time the performance was very unstable and the variance was extremely large, what could be the reason for this?

For example, In res14 dataset, sometimes the result is F1:0.70368, sometimes F1:0.41351

wuzhen247 commented 3 years ago

hi @BD-MF, thanks for your attention. The reason may be that the saved best model is overwritten by other experiments when you running several experiments at the same time. So you can rename the saved model with different datasets and rounds as the prefix.

BD-MF commented 3 years ago

The problem has been solved, thank you very much.

wuzhen247 commented 3 years ago

You're welcome.

JiahuiSophieHU commented 2 years ago

I can not find parameters about random seeds, maybe this can lead to unstable results as well.