search

hchiam / learning-prompt-eng

Learning Prompt Engineering
MIT License
5 stars 0 forks source link

guarantee that the randomWrapper doesn't exist in the userPrompt #2

Closed hchiam closed 9 months ago

hchiam commented 1 year ago

brainstormed possible vulnerabilities using ChatGPT, and came up with something along the lines of:

consider a while loop to regenerate the random number to make sure it doesn't exist in the userPrompt

https://github.com/hchiam/learning-prompt-eng/blob/main/example_prompts/helpers/antiIgnoreInstructions.js

(but this might hopefully get mitigated with the new ChatGPT setup where you have a separate input box for user input)

(aside: in a more complex app, consider encoding the input just like you would for untrusted HTML/JS code)

hchiam commented 9 months ago

there's settings in things like the messages object in gpt-4 apis and to me it seems still valid to use my extension of the random wrapper method (aka random sequence enclosure)