search

HITSZ-HLT / JointCL

18 stars 5 forks source link

The result of model #6

Closed clearpeck closed 1 year ago

clearpeck commented 1 year ago

1667890263118 1667890301570 1667890327031 1667890350763 1667890377895 1667890447902 1667890531426 For the doubt of the experimental results, the experimental results are too different. I have tried various combinations of hyper-parameters. What is the reason?

clearpeck commented 1 year ago

In addition, TOAD needs a lot of unlabeled data based on test topics to train topic discriminators. The author uses TOAD to do experiments on VAST and WT-WT datasets without unlabeled data. Is this setting reasonable?

BinLiang-NLP commented 1 year ago

1667890263118 1667890301570 1667890327031 1667890350763 1667890377895 1667890447902 1667890531426 For the doubt of the experimental results, the experimental results are too different. I have tried various combinations of hyper-parameters. What is the reason?

Hi, I apologize for the inconvenience. We have fixed the problems. Please run "git pull" to update the code. Please tune the parameter "--seed" for better performance since the small dataset. Please let me know if there is any problem. Thank you!!!

BinLiang-NLP commented 1 year ago

In addition, TOAD needs a lot of unlabeled data based on test topics to train topic discriminators. The author uses TOAD to do experiments on VAST and WT-WT datasets without unlabeled data. Is this setting reasonable?

Hi, we use the open-source code of TOAD to conduct experiments on VAST and WT-WT datasets. We do agree that using the unlabeled data may lead to better performance of TOAD. However, for VAST and WT-WT datasets, there are no corresponding unlabeled data and keywords. Therefore, we conducted a comparative experiment on the SEM16 dataset by using or removing the unlabeled data, and found that the performance gap is mostly within 5%, so the results on VAST and WT-WT datasets are overall reasonable.