search

datasec-lab / CodeBreaker

[USENIX Security '24] An LLM-Assisted Easy-to-Trigger Backdoor Attack on Code Completion Models: Injecting Disguised Vulnerabilities against Strong Detection
13 stars 1 forks source link

artifacts for LLM interaction #1

Open imnotkind opened 3 days ago

imnotkind commented 3 days ago

Hello, and congratulations on your excellent work! It has been a great source of inspiration for my own research, and I plan to cite it.

From what I gathered in your paper, you use LLMs in two key areas:

  1. Malicious code generation
  2. Malicious code detection

Since LLM interactions are often stochastic, I’ve found it challenging to replicate the exact results mentioned in the paper. I was wondering if you could provide any artifacts, such as logs of the input-output interactions with the LLMs, for both the code generation and detection phases? I tried looking in the repo but couldn't find them there.

Thank you for your help!

Jethro85 commented 1 day ago

Hello, and congratulations on your excellent work! It has been a great source of inspiration for my own research, and I plan to cite it.

From what I gathered in your paper, you use LLMs in two key areas:

  1. Malicious code generation
  2. Malicious code detection

Since LLM interactions are often stochastic, I’ve found it challenging to replicate the exact results mentioned in the paper. I was wondering if you could provide any artifacts, such as logs of the input-output interactions with the LLMs, for both the code generation and detection phases? I tried looking in the repo but couldn't find them there.

Thank you for your help!

Hi, thank you for your interest in our work. Some of the results related to code transformation for evading static analysis can be found in this folder. Unfortunately, we do not have logs of the experiments regarding code obfuscation to evade GPT detection. However, we have provided the code to replicate those results here.

Since this part of experiments were completed about a year ago, and GPT has undergone several upgrades since then, it might be challenging to replicate the exact outcomes as shown in the paper. Nevertheless, as long as the obfuscated code can bypass detection, the goal is achieved. You can also add personalized rules here to perform more advanced obfuscation.