search

jind11 / TextFooler

A Model for Natural Language Attack on Text Classification and Inference
MIT License
485 stars 79 forks source link

Doubt regarding synonym replacement #9

Closed RishabhMaheshwary closed 4 years ago

RishabhMaheshwary commented 4 years ago

I have doubt from the word replacement section of the paper.

  1. Are you querying the target model for every synonym of a word (selected after POS and semantic similarity check) and than selecting the synonym with least confidence score, and than repeating the process for each word in the text ordered by their classification drop score ?

  2. How are you calculating the no. of queries to the model ?

jind11 commented 4 years ago

Hi, here are the answers:

  1. Your understanding is all correct.
  2. Every time when I need to get the probability vector from the model by feeding the input sentence, it is counted as one query. For example, testing the confidence score after replacing the target word with every single word among the synonym candidates is counted as one query. Let me know if you have any other questions.
RishabhMaheshwary commented 4 years ago

So, Are you not counting the queries to the target model during word ranking step ?

jind11 commented 4 years ago

Actually any time that involves querying the model for the probability vector, I counted it in, which includes the word ranking step.

RishabhMaheshwary commented 4 years ago

Ok, Thanks.