Vulnerability Report: Insecure Output Handling in Lumos Plugin

[EDMPL]

Hi, sorry for reporting it here. I found a way to exfiltrate data from the user's chat because of the nature of the plugin that allows it to render most of the HTML tag and the default plugin RAG behavior. The issue occurs when a malicious website ask the local LLM model to injects an img tag via Indirect Prompt Injection into the content displayed by the plugin. This tag can trigger unauthorized data transmission to an external server controlled by the attacker when the user is actively chatting through the plugin while visiting the malicious site. I think most of the models will gladly put a simple image tag to the response, but so far I've just tried it with Llama 3.2 and Mistral.

Tested Version: 1.0.16 (Latest)

Vulnerability Description:

Lumos allows users to interact with a local LLM through a chat interface. During the default configuration usage, the plugin may render output from the current page that user is visiting in. However, when a malicious website is open, it can exploit the plugin's failure to sanitize output properly. Specifically, by embedding an HTML tag with a source pointing to the attacker’s server, the malicious site can capture sensitive data from the user's chat interactions and transmit it to an external location without user consent.

Impact

This vulnerability can lead to unauthorized exfiltration of sensitive information from the user's LLM chat, which could include confidential conversations, sensitive personal information, or other data handled during the plugin's use.

Exploitation Requirements: The attacker must host a malicious site and lure the user to visit it while they are actively using Lumos for LLM-based chat. As long as the 'Disable Content Parsing' feature is not enabled, the user can just ask anything while its in the malicious website tab for the exploitation to be successful.

Simple PoC

The attacker could hide the malicious prompt as seen in the screenshot below.

In a successful scenario, after user asks something, the plugin will try to read content of the current website, then the hidden prompt will be read and executed by the model. But with different models behaviors and unpredictability of the answer from AI, a payload (hidden prompt) will not be able to work in all model, a little bit customization may be required to the payload for it to work to specific model.

Like stated before, the exfiltration will be done via a rendered image tag like below.

Recommendation for Mitigation/Remediation:

1. Implement proper output sanitization in the Lumos plugin to ensure that HTML tags, especially <img>, are escaped before being rendered.
2. If possible, enforce strict content security policies (CSP) to prevent external sources from being injected into the plugin’s context.

Thanks.

[andrewnguonly]

Thanks for reporting the issue! I will look into it.

[EDMPL]

Hi Andrew. Just want to check since I'm not too familiar with browser extension. If you want to implement the input / output validation in the plugin, where and what will the fix be? Do you have to manually code for the input sanitization in the text box or is there simple way to do it? I tried to look at the javascript file but can't identify where the fix should be implemented. Thanks.

[andrewnguonly]

I'm not sure what the solution will be yet. The issue is related to the Markdown component, which is used to render the input/output: https://github.com/andrewnguonly/Lumos/blob/main/src/components/ChatBar.tsx#L724-L733

[EDMPL]

I see, thanks for the information. I see that you're using markdown-to-jsx library, I think you can just use the disableParsingRawHTML: true to stop it from rendering HTML tags in input and output of the text box. Other alternative is to encode certain characters like <, /, >, etc.

References: https://github.com/quantizor/markdown-to-jsx/pull/278 https://github.com/quantizor/markdown-to-jsx/pull/278/commits/852e80fe1f92274cacb622bc51c4430531861a49