search

jind11 / TextFooler

A Model for Natural Language Attack on Text Classification and Inference
MIT License
485 stars 79 forks source link

full adversarial example for FAKE using BERT #52

Closed maverickyu closed 2 years ago

maverickyu commented 2 years ago

can you give me the whole adversarial dataset after TextFooler Attack BERT ? Thank you so much

jind11 commented 2 years ago

Here it is: https://drive.google.com/drive/folders/12yeqcqZiEWuncC5zhSUmKBC3GLFiCEaN

maverickyu commented 2 years ago

Hi, Thanks a lot for your reply, and I have a question about the bert FAKE dataset. I saw the data only 47 KB and very few News lines in that,is that all testing adversarial data in your experiment ?

On Fri, Mar 11, 2022 at 2:49 AM Di Jin @.***> wrote:

Closed #52 https://github.com/jind11/TextFooler/issues/52.

— Reply to this email directly, view it on GitHub https://github.com/jind11/TextFooler/issues/52#event-6220002471, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIKVC55O5CHCU23O5W52HBDU7I73RANCNFSM5QL5IL6Q . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you authored the thread.Message ID: @.***>

maverickyu commented 2 years ago

Dear Sir, cause that I have to make a robust bert to defense attack such as TextFooler, there's still some question below : (1) I have my own train bert model for FAKE dataset, Is there any way to use your code to generate adversarial samples base on my model? (2) In yours attack_classification.py code, I have no idea what is the parameter counter_fitting_embeddings_path, can you explain it to me?

Thanks a lot.

On Fri, Mar 11, 2022 at 9:28 AM Maverick Yu @.***> wrote:

Hi, Thanks a lot for your reply, and I have a question about the bert FAKE dataset. I saw the data only 47 KB and very few News lines in that,is that all testing adversarial data in your experiment ?

On Fri, Mar 11, 2022 at 2:49 AM Di Jin @.***> wrote:

Closed #52 https://github.com/jind11/TextFooler/issues/52.

— Reply to this email directly, view it on GitHub https://github.com/jind11/TextFooler/issues/52#event-6220002471, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIKVC55O5CHCU23O5W52HBDU7I73RANCNFSM5QL5IL6Q . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you authored the thread.Message ID: @.***>

jind11 commented 2 years ago

Hi, Thanks a lot for your reply, and I have a question about the bert FAKE dataset. I saw the data only 47 KB and very few News lines in that,is that all testing adversarial data in your experiment ? … On Fri, Mar 11, 2022 at 2:49 AM Di Jin @.> wrote: Closed #52 <#52>. — Reply to this email directly, view it on GitHub <#52 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIKVC55O5CHCU23O5W52HBDU7I73RANCNFSM5QL5IL6Q . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub. You are receiving this because you authored the thread.Message ID: @.>

Yes, there are 1000 samples sampled from the original test set and used for our experiment to create adversarial samples.

jind11 commented 2 years ago

Dear Sir, cause that I have to make a robust bert to defense attack such as TextFooler, there's still some question below : (1) I have my own train bert model for FAKE dataset, Is there any way to use your code to generate adversarial samples base on my model? (2) In yours attack_classification.py code, I have no idea what is the parameter counter_fitting_embeddings_path, can you explain it to me? Thanks a lot. … On Fri, Mar 11, 2022 at 9:28 AM Maverick Yu @.> wrote: Hi, Thanks a lot for your reply, and I have a question about the bert FAKE dataset. I saw the data only 47 KB and very few News lines in that,is that all testing adversarial data in your experiment ? On Fri, Mar 11, 2022 at 2:49 AM Di Jin @.> wrote: > Closed #52 <#52>. > > — > Reply to this email directly, view it on GitHub > <#52 (comment)>, or > unsubscribe > https://github.com/notifications/unsubscribe-auth/AIKVC55O5CHCU23O5W52HBDU7I73RANCNFSM5QL5IL6Q > . > Triage notifications on the go with GitHub Mobile for iOS > https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 > or Android > https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub. > > You are receiving this because you authored the thread.Message ID: > @.***> >

(1) Yes, you can use this source code to generate adversarial samples to any model that has been trained on a certain dataset. (2) The counter-fitted embedding file is downloaded here: https://drive.google.com/file/d/1bayGomljWb6HeYDMTDKXrh0HackKtSlx/view

maverickyu commented 2 years ago

Thanks for your reply again, I thought that by any chance can I have the whole adversarial samples you use TextFooler to generate base on Bert model, maybe 1000 adversary samples or more? In my case, I need more TextFooler samples to train my robust model, thank you again.

從我的iPhone傳送

Di Jin @.***> 於 2022年3月12日 上午2:16 寫道：

 Hi, Thanks a lot for your reply, and I have a question about the bert FAKE dataset. I saw the data only 47 KB and very few News lines in that,is that all testing adversarial data in your experiment ? … On Fri, Mar 11, 2022 at 2:49 AM Di Jin @.> wrote: Closed #52 <#52>. — Reply to this email directly, view it on GitHub <#52 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIKVC55O5CHCU23O5W52HBDU7I73RANCNFSM5QL5IL6Q . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub. You are receiving this because you authored the thread.Message ID: @.>

Yes, there are 1000 samples sampled from the original test set and used for our experiment to create adversarial samples.

— Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you authored the thread.

jind11 commented 2 years ago

hi, I am sorry that I only have the 1000 adversarial samples provided in this repo but you can easily use this source code to generate more if you want.

maverickyu commented 2 years ago

Dear Sir, when I start to run attack_classification.py, it returned the following error No Such File " /data/medg/misc/jindi/nlp/embeddings/counter-fitted-vectors.txt" . Can you tell me how to generate this file? And why should we give this file to generate adversarial samples? Thank you a lot.

[image: 14.03.2022_12.22.03_REC.png]

On Sun, Mar 13, 2022 at 3:10 PM Di Jin @.***> wrote:

hi, I am sorry that I only have the 1000 adversarial samples provided in this repo but you can easily use this source code to generate more if you want.

— Reply to this email directly, view it on GitHub https://github.com/jind11/TextFooler/issues/52#issuecomment-1066041443, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIKVC54SX2ADFPKIY63FNVLU7WIFNANCNFSM5QL5IL6Q . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you authored the thread.Message ID: @.***>

maverickyu commented 2 years ago

Dear Sir, Now I have some adversarial samples, Can you explain how to use these samples to test your bert Model for FAKE ? I want to see the bert model test Accuray by TextFooler attack data.

Thanks and Regards

On Mon, Mar 14, 2022 at 12:26 PM Maverick Yu @.***> wrote:

Dear Sir, when I start to run attack_classification.py, it returned the following error No Such File " /data/medg/misc/jindi/nlp/embeddings/counter-fitted-vectors.txt" . Can you tell me how to generate this file? And why should we give this file to generate adversarial samples? Thank you a lot.

[image: 14.03.2022_12.22.03_REC.png]

On Sun, Mar 13, 2022 at 3:10 PM Di Jin @.***> wrote:

hi, I am sorry that I only have the 1000 adversarial samples provided in this repo but you can easily use this source code to generate more if you want.

— Reply to this email directly, view it on GitHub https://github.com/jind11/TextFooler/issues/52#issuecomment-1066041443, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIKVC54SX2ADFPKIY63FNVLU7WIFNANCNFSM5QL5IL6Q . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you authored the thread.Message ID: @.***>

jind11 commented 2 years ago

What is "FAKE"? Do you mean the fake news dataset? Basically, you can directly evaluate a bert model that has been fine-tuned on the specific dataset on those adversarial samples you obtained.

maverickyu commented 2 years ago

Dear Sir, May I share my BertClassifier to you for helping me generate the adversarial samples by Textfooler?

Thank you so much

On Fri, Mar 18, 2022 at 12:56 PM Maverick Yu @.***> wrote:

Dear Sir, I dont know these two file (data_defense/counter-fitted-vectors.txt', 'data_defense/cos_sim_counter_fitting.npy ) came from, can you give some clues? I want to use my own data to generate those two files.

Thank you so much

On Wed, Mar 16, 2022 at 5:13 AM Di Jin @.***> wrote:

What is "FAKE"? Do you mean the fake news dataset? Basically, you can directly evaluate a bert model that has been fine-tuned on the specific dataset on those adversarial samples you obtained.

— Reply to this email directly, view it on GitHub https://github.com/jind11/TextFooler/issues/52#issuecomment-1068478728, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIKVC53BDFXB752BKMSCK73VAD4P7ANCNFSM5QL5IL6Q . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you authored the thread.Message ID: @.***>

maverickyu commented 1 year ago

Dear Sir, I dont know these two file (data_defense/counter-fitted-vectors.txt', 'data_defense/cos_sim_counter_fitting.npy ) came from, can you give some clues? I want to use my own data to generate those two files.

Thank you so much

On Wed, Mar 16, 2022 at 5:13 AM Di Jin @.***> wrote:

What is "FAKE"? Do you mean the fake news dataset? Basically, you can directly evaluate a bert model that has been fine-tuned on the specific dataset on those adversarial samples you obtained.

— Reply to this email directly, view it on GitHub https://github.com/jind11/TextFooler/issues/52#issuecomment-1068478728, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIKVC53BDFXB752BKMSCK73VAD4P7ANCNFSM5QL5IL6Q . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you authored the thread.Message ID: @.***>

jind11 commented 1 year ago

hi, they are from this github: https://github.com/nmrksic/counter-fitting .