ScottLogic / prompt-injection

Application which investigates defensive measures against prompt injection attacks on an LLM, with a focus on the exposure of external tools.
MIT License
13 stars 9 forks source link

Occasionally the modal dialogs can no longer be opened #862

Closed chriswilty closed 4 months ago

chriswilty commented 4 months ago

Bug report

Description

Occasionally, the application gets itself into a state in which modal dialogs no longer open. This is really difficult to reproduce as we have yet to spot any pattern; this in itself indicates it might be a timing / race-condition issue.

It is not to do with the React double-render in dev/strict mode, as Kira has also spotted this in Vite preview mode.

Some "close dialog" events trigger other dialogs to open, which seems a likely candidate for this bug. In particular, there is complexity and redundancy in the process to open the MissionInformation dialog, as it is triggered when

Some of these calls are duplicated, for example:

These are problematic, because it means we have two calls to close the dialog, and one or two calls to open the MissionInformation dialog, in quick succession. If these occur out of intended order, that might cause the app to get into a corrupt state. It's worth noting that the API docs for HTMLDialogElement.showModal() state an error is thrown if the dialog is already in the "open" state, but our app is not catching that error. It is not clear from the docs what might happen in that case, but it seems entirely plausible that the dialog simply stops functioning.

In any case, we can start by adding some debug logging statements whenever a dialog is opened or closed, to see if any events fire in an unexpected order. We could then try to reproduce the bug, and then remove the redundancy in function calls and see if we can still reproduce the bug.

Reproduction steps

As noted above, this is proving difficult to reproduce, but start by repeatedly switching between sandbox and level1 from the Welcome dialog.

Expected behaviour

Modal dialogs should always appear when clicking related buttons (see below).

Acceptance criteria

kclark-scottlogic commented 4 months ago

Moving back for timing issue where level 1 info flashes a fraction of a second before going to sandbox dialog and vice versa