search

tech-srl / adversarial-examples

Code for the paper: "Adversarial Examples for Models of Code"
MIT License
17 stars 12 forks source link

How do you really define where in the code to perform the attack ? #3

Open hungkien05 opened 1 year ago

hungkien05 commented 1 year ago

Dear authors, I noticed in your deadcode test dataset, you added the deadcode and pre-process them. So how do you define those attack points, in other words, where to add those deadcode (or which varname to change) ?

Thanks in advance !

urialon commented 1 year ago

Hi @hungkien05 , @noamyft wrote the code, but until he responds - if I remember correctly, the deadcode was an unused variable declaration. Maybe this unused variable had a special name that allows the attack to recognize it?

hungkien05 commented 1 year ago

Dear @urialon Thanks for your reply. But I don't think you fully understand my question. What I want to ask is that: In the paper, you didn't mention where in a code snippet that the attack is performed, e.g where in a code snippet to add a deadcode or which varname in a code snippet is chose to be changed. I also cannot find the answer after reading your project code. Then I noticed in your test dataset, the attacks are already added and I assume that these attacks were added randomly and the project only takes in those datasets and compare the outputs of them to the original (pre-attacked) outputs.

However, I feel really skeptical about my assumption. Therefore, I want to ask you, the authors, if my assumption is true, or there are some method you used to define where to add the attacks, to achieve higher effectiveness of the attack.

Thanks for reading this quite long question. Have a good day !

urialon commented 1 year ago

Hi @hungkien05 , Thanks again for your interest in our work.

As far as I remember:

  1. VarName - the variable to attack is chosen randomly. I am assuming that if we had selected which variable to attack, we could have gotten an even better attack success rate. So I think that choosing the variable to attack in a more clever way is an interesting open research direction.
  2. DeadCode - page 14 says:

    In all cases, we arbitrarily placed the dead code at the end of the input method, and used our attack to find a new name for the new (unused) declared variable. In our preliminary experiments we observed that placing the dead code anywhere else works similarly

Does that help? I'm not sure I understood all your questions. Best, Uri

hungkien05 commented 1 year ago

Thank you @urialon for your reply. That really answered all of my questions !