[Multiple Tasks] FEAT add attack modules from moonshot

[eugeniavkim]

Is your feature request related to a problem? Please describe.

Adding in attack modules from Project Moonshot that can be adapted as converters under pyrit.prompt_converter

Describe the solution you'd like

Directly porting over the technique from attack-modules from https://github.com/aiverify-foundation/moonshot-data?tab=readme-ov-file#attack-modules

In order to prevent duplicate work, we can use this task list below to check off completed attack modules as well as commenting on which attack you are working on adapting into PyRIT.

• [x] Charswap Attack #403
• [x] Colloquial Wordswap Attack #406
• [x] Homoglyph Attack #407
• [ ] #426
• [ ] #427
• [x] Malicious Question Generator #397
• [ ] Textfooler (@visirion07)
• [ ] Textbugger (@visirion07)
• [ ] #428
• [x] Violent Durian #398

[eugeniavkim]

I will take on the colloquial wordswap attack and mark it completed on the task list once completed👍

[visirion07]

I will "attack" Textfooler and Textbugger. WIll mark it completed once done.

[KutalVolkan]

Hi @eugeniavkim ,

I would like to work on Malicious Question Generator and Violent Durian.

I also took a look at the Toxic Sentence Generator and noticed that 22 files have been flagged as unsafe. Just wanted to check with you—is it still safe to proceed with this model, or should we apply the same approach used in the Malicious Question Generator as an alternative?

Here’s the link to the files I mentioned: Toxic Sentence Generator.

Looking forward to your thoughts!

---

• [x] Malicious Question Generator

[romanlutz]

@KutalVolkan go ahead! Which files are unsafe?

[KutalVolkan]

@KutalVolkan go ahead! Which files are unsafe?

Hello Roman,

Here’s the link and the screenshot I mentioned regarding the unsafe files: Toxic Sentence Generator on Hugging Face.

[KutalVolkan]

Hello @romanlutz,

A few additional questions:

1. Should we create a PR for each converter individually, e.g., for the Malicious Question Generator, or should we wait until all the above attack modules from Project Moonshot are finished before submitting the PR?

Submitting separate PRs might allow for more focused reviews and quicker feedback on each converter, but I'll defer to your preference on how you'd like to handle it.

2. Regarding Violent Durian, I initially thought it would function more like a strategy inside the Red Teaming Orchestrator. Upon further review, I see that it operates more dynamically by convincing the LLM (prompt target) to take on a criminal persona. The setup involves a multiturn agent that manipulates the LLM into gradually adopting the identity of a criminal (e.g., Zodiac Killer, Ted Bundy) and generating responses as if it were that persona.

This contrasts with a standard converter that mostly modifies the input prompt. In this case, Violent Durian seems to guide a multi-turn conversation, progressively influencing the LLM to respond unethically and act in alignment with the persona.

For example, I plan to integrate this behavior into the Red Teaming Orchestrator by dynamically selecting a criminal persona and applying it to the conversation objective in the YAML-based attack strategy, adapting the YAML to fit the Violent Durian use case.

If you have a different approach or best practices to suggest, I’d be happy to incorporate them. Looking forward to your thoughts 😀

[romanlutz]

Yes, individual PRs are preferable, unless you're reusing pieces. Even then it's probably better to have them one after the other.

Your idea to use it on the orchestrator level makes sense. Essentially, this would be a new custom attack strategy.

[romanlutz]

@KutalVolkan go ahead! Which files are unsafe?

Hello Roman,

Here’s the link and the screenshot I mentioned regarding the unsafe files: Toxic Sentence Generator on Hugging Face.

Good question...

I have not used them before, but this sounds suspicious. Maybe it's because they're binary? I suppose we could go back to the paper and check how they generated these but that could involve a lot of work. Otherwise, I'm inclined to skip. Don't want to be responsible for making your machine unsafe 😆

[nina-msft]

Marking this with good first issue. The remaining tasks of:

• Insert Punctuation Attack
• Job Role Generator
• Toxic Sentence Generator

may be good first issues to tackle.

[nina-msft]

@visirion07 - are you still planning on taking a look at Textfooler and Textbugger? 😄

[visirion07]

Yes @nina-msft. Sorry got held up in some other work. Taking this up as a high priority. WIll post an ETA soon