search

reddy-lab-code-research / CodeAttack

Code for the AAAI 2023 paper "CodeAttack: Code-based Adversarial Attacks for Pre-Trained Programming Language Models
25 stars 6 forks source link

readme

CodeAttack: Code-based Adversarial Attacks for Pre-Trained Programming Language Models

This repository contains the code for the AAAI 2023 paper CodeAttack: Code-based Adversarial Attacks for Pre-Trained Programming Language Models.

Overview

Pre-trained programming language (PL) models (such as CodeT5, CodeBERT, GraphCodeBERT, etc.,) have the potential to automate software engineering tasks involving code understanding and code generation. However, these models operate in the natural channel of code, i.e., they are primarily concerned with the human understanding of the code. They are not robust to changes in the input and thus, are potentially susceptible to adversarial attacks in the natural channel. We propose, CodeAttack, a simple yet effective blackbox attack model that uses code structure to generate effective, efficient, and imperceptible adversarial code samples and demonstrates the vulnerabilities of the state-of-the-art PL models to code-specific adversarial attacks. We evaluate the transferability of CodeAttack on several code-code (translation and repair) and code-NL (summarization) tasks across different programming languages. CodeAttack outperforms state-of-the-art adversarial NLP attack models to achieve the best overall drop in performance while being more efficient, imperceptible, consistent, and fluent.

Run

To install the dependencies please execute the command pip install -r requirements.txt. To run the code, please execute python codeattack.py with the following arguments:

Argument Description
--attack_model Model that attacks
--victim_model Model to attack
--task The task to attack
--lang The input programming language dataset [java_cs, cs_java, java_small]
--use_ast A boolean flag for whether to use AST constraint
--use_dfg A boolean flag for whether or not to use DFG as a constraint
--out_dirname The output directory
--input_lang The input programming language (only required for GraphCodeBERT)
--use_imp A boolean flag to either attack random words or attack only important/vulnerable words
--theta The percentage of tokens to attack

Code for this repository has been adapted from CodeXGLUE, [CodeBERT)[https://github.com/microsoft/CodeBERT], and TextFooler.

Citation

@article{jha2023codeattack,
  title={Codeattack: Code-based adversarial attacks for pre-trained programming language models},
  author={Jha, Akshita and Reddy, Chandan K},
  journal={Proceedings of the AAAI Conference on Artificial Intelligence},
  year={2023}
  }