JianhuaSu
commented
8 months ago The following is my reproduction of the results. The DA-ADB result is generally higher than that of ADB, so there is nothing wrong with our model. The only difference is that the position of the bert model I use in the hyperparameter is
“uncased_L-12_H-768_A-12”.
Could it be a bert version issue? You can give it a try uncased_L-12_H-768_A-12
CodingPerson
Is the example found in open_intent_detection/examples/run_DA-ADB.sh meant to reproduce the published results? Running this and comparing to ADB gives significantly worse results. I took a look to see why this is briefly but didn't find an obvious answer.
The average of ADB across the experiments gave 84% and DA-ADB gave 47%
Example of ADB results from which averages of Acc where calculated
The only change was the location of the bert model in hyper-parameters which was the same for both methods:
'bert_model': "bert-base-uncased", Instead of: 'bert_model': "/home/sharing/disk1/pretrained_embedding/bert/uncased_L-12_H-768_A-12/",
Thanks for your help!