Regarding Semantic Similarity

[RishabhMaheshwary]

In the code, you are computing the semantic similarity of the current perturbed input with the previous perturbed input rather than the orignal input. How this guarantees that the final adversarial text will be semantically similar to the orignal input ?

[jind11]

Hi, as mentioned by the algorithm described in the paper and in the code as shown by this line: https://github.com/jind11/TextFooler/blob/0bb6bfbff97e72a41d33690a1857898e5332d335/attack_classification.py#L265, we have calculated the similarity score between the new adversarial text and the original text.

[RishabhMaheshwary]

But in Line 291 of same file we are updating the text_cache with text_prime. So I think we are matching it with the previous perturbed , or am I missing something ? Anyways this kind of semantic similarity method also seems fine if we follow the equivilance relation.

[jind11]

You are right, due to the line 291, the similarity score is calculated between the text perturbed in the last iteration and that in the current iteration. Using the equivalence chain rule, it is ok. Comparing the new text with the original text should be better. Thank you for this finding!

[RishabhMaheshwary]

Actually, I tried comparing new_texts with tex_ls i.e the orignal text but that does not work that well. May be for that we need to change threshold values to below 0.7.

Thanks anyways.

[jind11]

I agree, in my experiments, I also found 0.7 is a very harsh threshold, and 0.5 is much better.