search

bogdan-kulynych / textfool

Plausible looking adversarial examples for text classification
MIT License
92 stars 32 forks source link

The result does not match. #2

Open sharejing opened 5 years ago

sharejing commented 5 years ago

Hi! @bogdan-kulynych Thanks for your nice job! I have a puzzle when I run paraphrase.py(not use typos):

Original text: Soft skills like sharing and negotiating will be crucial. He says the modern workplace, where people move between different roles and projects, closely resembles pre-school classrooms, where we learn social skills such as empathy and cooperation.

Perturbed text: subdued skills like sharing and negotiating will be crucial . He says the modern workplace , where people move between different roles and projects , closely resembles pre - school classrooms , where we learn social skills such as empathy and cooperation .

Why is it different from the result on the picture?

paraphrase_example

bogdan-kulynych commented 5 years ago

Hi! Since the search process itself is deterministic, should be because the difference in our trained models.

sharejing commented 5 years ago

Hi! Since the search process itself is deterministic, should be because the difference in our trained models.

Sorry! I don't understand you. Does it need a pre-trained model when I run paraphrase.py?

bogdan-kulynych commented 5 years ago

Ah, I misunderstood. You don't need a pre-trained model to run python paraphrase.py.

The example on the picture shows an adversarial example that evades the model. Running python paraphrase.py does not run the attack, it just shows an example of paraphrases. To reproduce the picture, you want to do python run_demo.py.

What is strange though, is that by default paraphrases include typos, and yours did not.