Open milumilum opened 3 years ago
I am also not sure why there is such a huge change. Did yo use the same BERT model parameters that I provided?
Yes, the model parameters and data are all provided by you.All the other attacks are close to the results given in your paper, only this one deviates a lot, so I wonder whether it is due to the data.
Where is the emdding.npz file, please? Or how is it generated?
hi, when I experimented with the 1000 samples you provided from the AG dataset and the bert model, the results are:
For dataset data/ag: For target model bert: original accuracy: 94.200%, adv accuracy: 29.500%, avg changed rate: 25.076%, num of queries: 481.7
But the adv accuracy in your paper is 11.5%. I'm not quite sure what led to this result. So did you use the same 1000 examples in your paper?Thanks.