search

BoxcarsAI / boxcars

Building applications with composability using Boxcars with LLM's. Inspired by LangChain.
MIT License
430 stars 39 forks source link

security #40

Closed lucasluitjes closed 1 year ago

lucasluitjes commented 1 year ago

I wanted to give you a heads up but there doesn't seem to be a way to reach the boxcars team privately. No email address or contact form, and the Twitter account doesn't accept DMs. There is no listing of production apps using this library, so it seemed unlikely that end-users would be harmed as a result of my post, so I just went ahead. Realized later I could create a github issue here. Feel free to delete.

Some of the features (activerecord and sql in particular) have big security implications that are completely undocumented. This seems to be a trend in LLM composability frameworks, so I wrote a post about it. Details here. If you improve security or make the risks clear, I can update the post to reflect that.

francis commented 1 year ago

Hi Lucas (@lucasluitjes),

You are absolutely right about the concerns you have raised.

Boxcars is currently in alpha version 0.2.2 and is meant to be used by an Admin user who could hack the system in many easier ways. As you’ve said, we don’t call this out. We will get the Readme updated right away!

We did spend a little time on Boxcars::ActiveRecord to make it read-only by default, but there is more to be done here to make it “prompt hacking proof.” Boxcars::SQL can reuse some of this logic, and we will add an issue to track this.

For more context: we started working on Boxcars after playing with the Python package LangChain and realizing that you don’t need Python to get these concepts to work. We haven’t looked extensively, but we’d assume that LangChain would have similar security concerns.

We think that once we get the basics working, we’ll incorporate a sandbox for the execution of the code.

As far as not seeing an email for us, we need to come up with a better way to advertise this while still reducing the amount of SPAM we get - it is: hi@boxcars.ai . You can find it in our ruby gem specification here https://github.com/BoxcarsAI/boxcars/blob/main/boxcars.gemspec

So in recap, this is what we will do:

  1. Update the Readme to be more explicit on this topic.
  2. We will write a blog post highlighting the security concerns and invite discussion on ways to address them.
  3. Create new Issues so that some of these can be addressed.

We’d love to connect with you and get more feedback if you’re open to it.

lucasluitjes commented 1 year ago

Thanks for the quick reply!

I think langchain has similar issues (see here for example). I assume the JS port also has these problems. Didn't mean to single you out but I didn't have time to make PoCs for all of them. And I'm more familiar with Ruby, so I looked at Boxcars first.

I'll update the post to reflect that you're working on it. Saw the email, lets connect there.