sherdencooper / PromptFuzz

MIT License
11 stars 2 forks source link

error during bash ./Scripts/promptfuzz_focus_attack.sh. #37

Open ambarion opened 1 month ago

ambarion commented 1 month ago

'parent_prompt' cannot be locate in csv. Did it mean 'parent' or something else?

Traceback (most recent call last): File "/data2/ambarion/workspace/PromptFuzz/./Experiment/run.py", line 41, in run_fuzzer(args) File "/data2/ambarion/workspace/PromptFuzz/PromptFuzz/Fuzzer/promptfuzz.py", line 151, in run_fuzzer fuzzer.run() File "/data2/ambarion/workspace/PromptFuzz/PromptFuzz/Fuzzer/gptfuzzer/fuzzer/core.py", line 142, in run mutated_results, mutator_name = self.mutate_policy.mutate_single(seed) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/data2/ambarion/workspace/PromptFuzz/PromptFuzz/Fuzzer/gptfuzzer/fuzzer/mutator.py", line 410, in mutate_single few_shot_original = few_shot_prompt['parent_prompt'].tolist()


  File "/home/ambarion/anaconda3/envs/factory/lib/python3.12/site-packages/pandas/core/frame.py", line 4102, in __getitem__
    indexer = self.columns.get_loc(key)
              ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ambarion/anaconda3/envs/factory/lib/python3.12/site-packages/pandas/core/indexes/base.py", line 3812, in get_loc
    raise KeyError(key) from err
KeyError: 'parent_prompt'
ambarion commented 1 month ago

I think the mistake is caused in https://github.com/sherdencooper/PromptFuzz/blob/5c8fe3713cc6984465c068e9bedccbcdb5dbf41c/PromptFuzz/Fuzzer/gptfuzzer/fuzzer/mutator.py#L410. Clearly, the 'parent_prompt' are not record during https://github.com/sherdencooper/PromptFuzz/blob/5c8fe3713cc6984465c068e9bedccbcdb5dbf41c/PromptFuzz/Fuzzer/gptfuzzer/fuzzer/core.py#L192, the writerow only record prompt_node.parent.index, rather than prompt_node.parent. So the few shot attack in https://github.com/sherdencooper/PromptFuzz/blob/5c8fe3713cc6984465c068e9bedccbcdb5dbf41c/Scripts/promptfuzz_focus_attack.sh#L11 cannot be done. Could you please refine the process in focus attack stage? Looking forward to your attention.

AndyShaw01 commented 1 month ago

Thank you for your detailed feedback! The issue you pointed out was indeed caused by the omission of the parent_prompt field during the generation of the few_shot_example. I have updated the relevant script to fix this problem.

To resolve the issue, please follow these steps:

  1. Pull the latest code updates.
  2. Replace the original script and regenerate the few_shot_example data by executing the following command: bash ./Scripts/analysis_preparation.sh

This will ensure that the parent_prompt is correctly recorded in the output CSV.