ScottLogic / prompt-injection

Application which investigates defensive measures against prompt injection attacks on an LLM, with a focus on the exposure of external tools.
MIT License
16 stars 10 forks source link

Stop adding messages to backend history via frontend πŸ› πŸ›  #828

Open pmarsh-scottlogic opened 8 months ago

pmarsh-scottlogic commented 8 months ago

I note that there is an endpoint to add a message to the backend's chat history. We only every do this as a direct result of something that happened in the backend. Therefore this endpoint and all its associated network calls don't need to exist.

πŸ”§ Remove the endpoint /openai/addHistory and add messages to history in the backend instead. Delete the frontend methods that use this endpoint, including addInfoMessageToChatHistory, addResetMessage, addInfoMessage, and other methods tat only exist to add stuff to backend history.

The user actions that would result in a change in chat history and thus a superfluous API call are:

ACCEPTANCE CRITERIA

The following acceptance criteria boil down to doing the above listed actions, checking that the actions work and the info messages are added, and there is only one network call

WHEN user wins a level THEN level completion message appears as normal in chat AND only one network call of type fetch

WHEN user sends a message that triggers a defence (e.g. configure character limit to be super small, then send a long message) THEN the defence triggered message appears as normal in chat AND only one network call of type fetch

WHEN user sends a message that alerts a defence (e.g. configure a character limit to be small, then disable the defence again and send a long message) THEN the defence alerted message appears as normal in chat AND only one network call of type fetch

WHEN user configures a defence (i.e. changes the character limit) THEN the defence configured message appears as normal in chat AND only one network call of type fetch

WHEN user toggles a defence on or off THEN the defence enabled / disabled message appears as normal in chat AND only one network call of type fetch

WHEN user changes the chat model (under model configuration) THEN the model changed message appears as normal in chat AND only one network call of type fetch

WHEN user changes a configuration of the chat model (under model configuration) THEN the model configured message appears as normal in chat AND only one network call of type fetch


Here's how you quickly find out what api calls are being made that are fetch type. Note that the CombinedFonts call is of type xhr, therefore we ignore it.

image

chriswilty commented 5 months ago

Three usages remaining:

Once these have been tackled, the service call can be removed from frontend and backend code.