Closed sullendhy closed 1 year ago
Thank you for your interest in our paper!
Query refers to the number of black-box queries to the target model before successfully attacking it. This is only relevant for the transfer attack experiment. You can extract the number of queries from the result of the evaluate_adv_samples.py
as follows.
all_corr
in https://github.com/facebookresearch/text-adversarial-attack/blob/main/evaluate_adv_samples.py#L216. The (i,j)th entry indicates whether the model correctly classified the j-th adversarial example for the i-th clean sample or not.all_corr
happens at (i,j) then (j+1) queries were needed to successfully attack the sample. You can compute a mean over a batch of samples to get an average query count.Hope this helps.
Thank you for your wonderful job and code. when I try to reproduce your result, i can not find the result of "query" metric. So what does the "query" metric mean in the paper. And i will appreciate it if you can tell me the specific implementation of calculating "query". Thanks again for your nice job !