Open ambarion opened 1 month ago
I think the mistake is caused in https://github.com/sherdencooper/PromptFuzz/blob/5c8fe3713cc6984465c068e9bedccbcdb5dbf41c/PromptFuzz/Fuzzer/gptfuzzer/fuzzer/mutator.py#L410. Clearly, the 'parent_prompt' are not record during https://github.com/sherdencooper/PromptFuzz/blob/5c8fe3713cc6984465c068e9bedccbcdb5dbf41c/PromptFuzz/Fuzzer/gptfuzzer/fuzzer/core.py#L192, the writerow only record prompt_node.parent.index, rather than prompt_node.parent. So the few shot attack in https://github.com/sherdencooper/PromptFuzz/blob/5c8fe3713cc6984465c068e9bedccbcdb5dbf41c/Scripts/promptfuzz_focus_attack.sh#L11 cannot be done. Could you please refine the process in focus attack stage? Looking forward to your attention.
Thank you for your detailed feedback! The issue you pointed out was indeed caused by the omission of the parent_prompt
field during the generation of the few_shot_example
. I have updated the relevant script to fix this problem.
To resolve the issue, please follow these steps:
bash ./Scripts/analysis_preparation.sh
This will ensure that the parent_prompt
is correctly recorded in the output CSV.
'parent_prompt' cannot be locate in csv. Did it mean 'parent' or something else?
Traceback (most recent call last): File "/data2/ambarion/workspace/PromptFuzz/./Experiment/run.py", line 41, in
run_fuzzer(args)
File "/data2/ambarion/workspace/PromptFuzz/PromptFuzz/Fuzzer/promptfuzz.py", line 151, in run_fuzzer
fuzzer.run()
File "/data2/ambarion/workspace/PromptFuzz/PromptFuzz/Fuzzer/gptfuzzer/fuzzer/core.py", line 142, in run
mutated_results, mutator_name = self.mutate_policy.mutate_single(seed)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data2/ambarion/workspace/PromptFuzz/PromptFuzz/Fuzzer/gptfuzzer/fuzzer/mutator.py", line 410, in mutate_single
few_shot_original = few_shot_prompt['parent_prompt'].tolist()