FirebaseExtended / bolt

Bolt Compiler (Firebase Security and Modeling)
Apache License 2.0
897 stars 107 forks source link

Bolt for Cloud Firestore #216

Open SamyPesse opened 7 years ago

SamyPesse commented 7 years ago

Even if the new Cloud Firestore has a better rules language than the RTDB, it still looks limited to write easy to understand document types.

It'd be awesome to use the Bolt language to write Cloud Firestore rules.

wcandillon commented 6 years ago

I bootstrapped a generator for firestore at https://github.com/wcandillon/firebase-bolt-compiler/pull/18. Nothing working yet but just to let you know. The semantic of Bolt doesn't appear to match 100% the one of firestore but I'm trying see if the generator can work for small use cases. The operators seem also different.

SamyPesse commented 6 years ago

@wcandillon Awesome 👍

Yes, some changes will be needed to the semantic, for example:

collection /users {
    document /{userID} is User {
        collection /tweets {
            document /{tweet} is Tweet
        }
    }
}

type User {
    name: String
}

type Tweet {
    message: String,
    postedAt: Date
}
wcandillon commented 6 years ago

@SamyPesse I implemented a small use case with firestore and I think it can be done with Bolt, I will keep you posted.

rockwotj commented 6 years ago

I believe that Cloud Firestore's rules system is supposed to be loosely modeled after Bolt, I don't know if we're looking to support anything here for Cloud Firestore. What does Bolt provide that the current rules language doesn't?

SamyPesse commented 6 years ago

@rockwotj Here are the main reasons we prefer Bolt:

For example, this is easy to read, understand and extend:

type User {
   name: ShortString,
   age: Number
}

type BlogPost {
    title: ShortString,
    postedBy: UserID,
    body: PostBody,
    deletedAt: Date | Null
}

type UserID extends String {
    validate() { root.users[this] != null }
}

type ShortString extends String {
    validate() { this.length < 50 }
}

type PostBody extends String {
    validate() { this.length < 500 }
}

And this is not (and yet it's still a small example without write rules):

service cloud.firestore {
  match /databases/{database}/documents {
    match /users/{userId} {
        allow read: if resource.data.keys().hasAll(['name', 'age'])
              && resource.data.size() == 2
              && resource.data.name is string
              && resource.data.age is int
              && resource.data.name.length < 50;
    }
    match /posts/{postId} {
        allow read: if resource.data.keys().hasAll(['title', 'postedBy', 'body'])
              && (
                resource.data.size() == 3 ||
                resource.data.size() == 4
              )
              && resource.data.title is string
              && resource.data.title.length < 50
              && resource.data.postedBy is string
              && exists(/users/${resource.data.postedBy})
              && resource.data.body is string
              && resource.data.body.length < 500
              && (resource.data.deletedAt ? resource.data.deletedAt is date : true);
    }
  }
}

I'm a big fan of Firebase, and my company is currently investing in it. But I'm really worried about the focus you have on security. I agree that the Google infrastructure is 100% safe, and that it's possible to build secure applications on top of it, but you are not helping your users build secure applications:

A few firebase production applications that I checked out had obvious security flaws, it was possible to read/write almost everything in their database, I've contacted them to signal these flaws.

I understand that pushing too much documentation/tooling about security to beginners who are building prototypes may push off some of them, but there should be a way for Firebase to help serous companies build reliable and secure applications on top of it.

rockwotj commented 6 years ago

@SamyPesse I heard you loud and clear yesterday as for as bumping security concerns, I'm working with our Developer Relations and Documentation folks to improve things here for the Realtime Database (Better docs on best practices, examples of how to version and deploy rules across environments, etc). Firestore is also working on improving this, @mcdonamp can explain more on the efforts there. As for the rules example you gave, you can have functions in the new rules languages (see here), which I think will help make things more manageable/readable, but I do understand the appeal of having "types" at locations in your database.

So you could make your example the following:

function isUser(resource) {
  return resource.data.keys().hasAll(['name', 'age'])
              && resource.data.size() == 2
              && resource.data.name is string
              && resource.data.age is int
              && resource.data.name.length < 50;
}

function isPost(resource) {
    return resource.data.keys().hasAll(['title', 'postedBy', 'body'])
              && (
                resource.data.size() == 3 ||
                resource.data.size() == 4
              )
              && resource.data.title is string
              && resource.data.title.length < 50
              && resource.data.postedBy is string
              && exists(/users/${resource.data.postedBy})
              && resource.data.body is string
              && resource.data.body.length < 500
              && (resource.data.deletedAt ? resource.data.deletedAt is date : true)
}

service cloud.firestore {
  match /databases/{database}/documents {
    match /users/{userId} {
        allow read: if isUser(resource);
    }
    match /posts/{postId} {
        allow read: if isPost(resource);
    }
  }
}

Thanks for your input! That's super helpful to hear.

SamyPesse commented 6 years ago

I heard you loud and clear yesterday as for as bumping security concerns

@rockwotj That was not me, That must have been Aaron, my co-founder :)

Thank you for your answer! Let us know if you need some infos on how our team is using Firebase. As I said before, we are big fan of Firebase ❤️ that's why we may be harsh on feedback sometimes.

rockwotj commented 6 years ago

@SamyPesse Ah sorry I never caught his name, and I saw the GitBook connection, so thought it was you!

I love the feedback, it's not harsh, but I totally agree we need to do more in this area... stay tuned! If others have thoughts on this topic, I'd love to hear them as well.

rockwotj commented 6 years ago

(BTW, I just found that we have a simulator here: https://github.com/firebase/bolt/blob/master/src/simulator.ts, which might be useful for some of those tests you've mentioned, it does require hitting a real firebase database however. I will add some documentation for this in the near future.)

SamyPesse commented 6 years ago

Currently, we have setup 800 unit tests using Jest and targaryen, tests don't require a real database and look like:

expect({ uid: 'foo' }).canRead('/users/someone');
expect({ uid: 'foo' }).cannotRead('/users');

It works very well, but targaryen doesn't plan on supporting Firestore.

We've already setup a JS layer on top of the firebase database so that our application can be "database agnostic", and we can switch from the RTDB to Firestore as soon as we've updated the rules and the tests are passing.

Our current setup Bolt + Targaryen + RTDB works well, but we want to change it to be Bolt + SomeTestRunner + Firestore.

The application code is ready, but adapting more than 1300 lines of Bolt to the Firestore security rules syntax without being able to run unit tests is an impossible task.

Basically my biggest request for Firebase is not to support Firestore export for Bolt, but to release a JS assertion library for unit testing the new Firestore rules.

toddpi314 commented 6 years ago

Chiming in here to put my support behind a bolt compilation to Firestore Schema. Moreover, why not just support Bolt in the Firebase Console?

I dread migrating security rules forward and am hesitant to invest in another schema language (Firestore) that is a step backwards from Bolt.

Bitterness aside, Firestore Schema is a step in the right direction. Please keep up the good work evolving that language.

wcandillon commented 6 years ago

@rockwotj I like the current security language but it would be great if you could provide support Type support the same way firebase bolt does. Providing us with with parsers and tools for the firestore security language would be great since we can use it to generate other code artefacts (as described here). The testing part as mentioned by @SamyPesse is critical too.

I like firestore a lot ❤️ and I'm looking forward to see the updates on this side of things.

asciimike commented 6 years ago

(Product Manager for Rules here)

First off, thanks for all the feedback. As @rockwotj said, please keep it coming--it's how we improve the product.

To summarize the above, it sounds like there are a few requests to address Rules pain points:

I'll handle these in separate replies so we can reference them independently.

TristonianJones commented 6 years ago

@SamyPesse I can definitely see how types would be desirable for Firestore. Our preference has been to support this sort of feature in Firestore directly with the ability to import the types into the latest security rules so you could write the following:

import my.package.types;

service cloud.firestore {
  match /databases/{database}/documents {
    match /users/{userId} {
        allow read: if resource.data is types.User;
    }
    match /posts/{postId} {
        allow read: if resource.data is types.Post;
    }
  }
}

Type information is super useful for code-generation and schema enforcement, and because it's a separate concern (often with a much different life-cycle from authorization), it was not included in the latest version of rules.

We're working on the testing story. It needs a developer experience, and all of the public resources available to do testing are not documented.

asciimike commented 6 years ago

Better support for resource/schema definition, type validation

I've spent a long time thinking about this, and the short answer is that I think we need to more clearly separate resource definition from validation and authorization (and possibly separate the latter two as well).

I worry that it's too easy to break schema/validation/authorization when trying to change a different one. Especially on teams where one person may be responsible only for schema, while another may be responsible for authorization, we'll quickly run into visibility and access issues if they remain merged.

There are a number of standards we can look to for defining resources in an API:

As @TristonianJones said, there is the option to build this in to the language. I'd be curious if folks on this thread prefer the built-in version or a separated world.

The worry here is that we end up with N different configs, and thus doing anything requires a ton of effort. Any feedback on separating this, or which config to use, would be appreciated.

asciimike commented 6 years ago

Better tooling to identify, reproduce, and fix Rules issues

As @TristonianJones said, we're long overdue on better tooling here. I think there are three tools in particular that may be helpful here.

Unit Testing: Unbeknownst to all (because it's not documented), we have an API endpoint that allows for unit testing Rules. We're working on documenting this, and if folks are interested in taking a look at it earlier, let me know and I can provide you early access docs to look at. We'd appreciate some feedback here. Eventually, we aim to build tests into the console, though we haven't invested a ton of UX love into this.

Local (specifically offline) testing is a separate issue that we don't currently have a solution for yet, but I think we're laying the foundation for it (more on this in the next post).

Simulation: Simulation can be thought of as a request to fetch appropriate data (either from resource or in the case of Firestore a get()/exists()), followed by a single unit test. This is first going to show up in console (similar to the existing Realtime Database simulator), and may eventually work it's way into the CLI or a standalone library.

Static Analysis: The main use we've seen is informing other views with information about what's covered in Rules. For example, imagine the data viewer being able to show which data is appropriately secured, if the types align, etc.

We've also considered additional tools, such as canary deploys (run a new version of Rules on some % of requests and see if there's a difference in % of allows/denies, alert if this deviates significantly).

If y'all have other thoughts, I'd love to hear them as well.

asciimike commented 6 years ago

Building a community around Rules

We're already making moves to make the language more accessible to developers, including open-sourcing the spec for our expression language, with additional tooling that conforms to the spec forthcoming. We plan on open sourcing the Rules language spec and tooling as well.

Working in the open should accelerate developer adoption and lead to a better dialog and evolution of tools. This will allow developers to start building static analysis tools, syntax highlighters, local test environments, etc., even if the core Firebase team isn't focused on building them at that moment.

asciimike commented 6 years ago

@SamyPesse Your assert thread jogged my memory and reminded me that the same folks (waves to @goldibex) also did a chai package for asserts: https://github.com/casetext/chai-fireproof

return expect({ uid: 'metropolis:maria' }).can.read.ref(root.child('users/maria'));

I bet that this could be hacked together with the unit test API we provide to steer more in the direction you're asking for. Eventually, with some of the other tools open sourced, local evaluation should also be possible.

RyanWarner commented 6 years ago

Bolt syntax is significantly easier for me to read and understand. While the Firestore rule syntax is a big improvement over the Realtime Database, it's not not as clean and concise as Bolt. I intend to use both databases in my project, and it would be great to have one, unified rule language.

It sounds like Firebase is aware of most or all of these issues, but I just wanted to voice my thoughts. Thank you for these awesome libraries :)

WhatsThatItsPat commented 6 years ago

I'm addicted to Bolt. I can't imagine going back at this point.

I haven't had a chance to look at Firestore yet, but hearing that there isn't a similar way to create rules means I'll be delaying that look.

asciimike commented 6 years ago

@patrickmcd can you clarify what about Bolt it is you like? Syntax, features (like types), or something else?

willhlaw commented 6 years ago

we have an API endpoint that allows for unit testing Rules. We're working on documenting this, and if folks are interested in taking a look at it earlier, let me know and I can provide you early access docs to look at.

@mcdonamp, great responses / great work. I'm interested in those early access docs as I'm beginning to break ground on a new product that requires tight security.

asciimike commented 6 years ago

@willhlaw and others, here's a first draft of docs for using the unit test API.

We know the experience using the Google API explorer is a little jank--we'd love feedback on how you'd like to see this better incorporated (e.g. UI, CLI, SDK support, etc.). Also would love info on the syntax (e.g. prefer YAML over JSON, etc.).

bijoutrouvaille commented 6 years ago

@mcdonamp there is a piece that I think is missing from this discussion. Since Bolt is so similar to Typescript, all it takes is a few simple regex rules to parse the Bolt types into a Typescript file, to be used throughout a Typescript project. As you can imagine, having this kind of static type checking greatly increases the stability of software that relies on Firebase. I'm not sure I'd be willing to give up this kind of confidence, even for all the wonderful features that Firestore is promising.

asciimike commented 6 years ago

@bijoutrouvaille I believe this is what https://github.com/firebase/bolt/issues/216#issuecomment-338816562 is aiming towards. While Bolt types can easily translate to Typescript, OpenAPI and Proto/Thrift can generate clients for many different languages (e.g. swagger-codegen, protoc --$lang_out).

This can potentially be used to keep types on clients and types in Rules in lock-step, though it adds significant overhead for developers as they're first getting started. Being able to go the other way (start writing an app, export those types to Rules) seems like it's more valuable for the path many Firebase developers go through.

bijoutrouvaille commented 6 years ago

@mcdonamp my bad, I've never used these. If the trade-off is between supporting more languages at the cost of a sensible overhead and ability to export types from a select few, the first option seems preferable.

If the "overhead" reads clearly in complex projects, installs without severe headache and is well-documented, then I'm rooting for it. It would be particularly nice to have typed clients, which seems at least feasible in the first option. Thanks for all your effort!

asciimike commented 6 years ago

@bijoutrouvaille not your fault at all--we likely would have picked one of those if there was a clear winner in the space. Unfortunately, many devs (including a majority of Firebase folks) aren't aware of these tools and thus they present an extra barrier to entry.

I think we need a gradually typed system, where a developer can start building an app without learning a new syntax, but as they discover they need it (or want other benefits), they can add it on.

willfarrell commented 6 years ago

@mcdonamp great to see a lot of thought is being put into how to best address this. Thanks for leading. I totally agree the validation and authorization should be separated.

I've used/evaluated almost all the ones you listed in https://github.com/firebase/bolt/issues/216#issuecomment-338816562

Schema (API) These schemas are more typically used in how to structure data when responding to an API request.

Personally I found that JSON Schema is a solid choice, at least for javascript web applications.

Hope this is helpful on the validation side. When it comes to Authorization, every project seems to be different in it's requirements on how to handle this. @TristonianJones proposal for authorization (https://github.com/firebase/bolt/issues/216#issuecomment-338805421) looks pretty solid to me. Combine that with the ability to include a schema from the option of bolt / protocol buffers/ json schema would be really powerful. Alternatively a json schema plugin could be written to denote authorization.

import my.package.types;
import User from 'schema.user.json';

service cloud.firestore {
  match /databases/{database}/documents {
    match /users/{userId} {
        allow read: if (auth.uid is data.uid) or (auth.role is 'admin');
        schema json: User
    }
    match /posts/{postId} {
        allow write: if resource.data is types.Post;
        schema bolt: types.Post
    }
  }
}
adamduren commented 6 years ago

I had similar thoughts for separating authorization from validation and got pointed to this thread.

I'm a fan of JSON schema and have been using it to define my firestore schema. Using this library I've been able to output TS interfaces and have a small wrapper around the firestore API that types queries, docs, and collections. My current approach has shortcomings that I'd like to solve such as:

I was using a JSON schema library to generate validation functions from the schema file but there were additional validations I wanted to express like cross field constraints such as startDate <= endDate.

However from reading @willfarrell's post some of my issues might be solved with ajv along with it's plugins.

willhlaw commented 6 years ago

I have the start of a node npm package, firestore-security-tests, that can be used to test Firestore rules. It requires a credentials JSON file (such as the firebase-adminsdk service account creds) and a test Resource object with source and testSuite array as described in the draft Google Doc that @mcdonamp posted.

https://www.npmjs.com/package/firestore-security-tests

Hey @mcdonamp, it would be nice to be able to add a label or a name for each testSuite.testCase so that we can compare and nicely print out the result of the test.

asciimike commented 6 years ago

We've taken a first step on validation here. I've only published the README, but if folks here plan on using it, we're comfortable open sourcing the implementation. Ideally I'd like to get to the point where we also generate .js, .ts, .java, .h/.m, .swift models as well. I assume using JSON Schema (or any other IDL) to generate clients would also be possible. cc @adamduren/@willfarrell

@willhlaw, awesome, thanks for taking a stab at that (and honestly surprised that our API is exposed in the Google APIs package). Thanks for the thought, definitely agree that we need the ability to differentiate which unit tests succeed and which fail. Technically the results are returned in the order sent, so it's possible to print out exactly which tests pass and which fail. I think that this may not be an issue with a Mocha/Chai runner (which would provide such naming and pretty printing).

willhlaw commented 6 years ago

Looks good. Having a complete flow for validation looks closer now with the protobuf tooling.

@mcdonamp, where is the best place to discuss the first draft of the Google Doc for the firebaserules api? For example, is 'post' supported yet? I can't seem to construct a POST request to give a success on ALLOW when rule for write is simply if true. Also, where are rulesets defined because it seems this would be very useful to test against the actual deployed rules and is it an "in progress" feature or is there a working example?

asciimike commented 6 years ago

@willhlaw I added instructions to that doc on how to test on already released Rulesets (specifically on the current Release). We'll provide better docs on the API in the near future (once it's actually public, currently we treat it as an implementation detail).

Almost all Google API methods are POST requests, with a custom method (:customMethod) appended to the request. You should be able to copy/paste the output of the Google API explorer curl request, then add the appropriate auth token.

willhlaw commented 6 years ago

@mcdonamp, thanks. The Rulesets make more sense now and I see a way forward. They must be catalogued internally within Google's infrastructure and not exposed (yet) in the Firebase console. I'll try to add functionality to get the current ruleset from the api to the starter project I mentioned above, firestore-security-rules.

But to my question on POST requests, I should have said that I cannot get a simple testCase using method 'post' to work successfully. I have created a minimal reproduction of the issue here using runkit. See that the get succeeds and the post tests with and without a resource fail. I'd love to know what I'm doing wrong. Any help or a screenshot of a real life example posted to that Google Doc would be greatly appreciated!

asciimike commented 6 years ago

@willhlaw FYI you're going to want to delete the private key from that example (as well as delete the key in the Console and re-create a new private key).

asciimike commented 6 years ago

Not secret, just haven't had time to externalize this stuff in a sensible way. We are working to expose version history/rollback in the Console in the near future.

Here's the TL;DR: of Rules...

Firebase Rules is composed of two primitives:

This allows us to maintain a version history (e.g. listRulesets API), as well as support many different products (you'll notice we have Releases for Firebase.Storage as well as Cloud.Firestore).

Release naming also provides resource binding, and can creatively be used to support percent rollout or A/B testing.

willhlaw commented 6 years ago

@mcdonamp, I realize that about the private key. I almost put a comment to that effect in the sample about why it was being disclosed. It's creds to a firebase project set up to help facilitate testing for firebase-security-store and would be fine with any abuse (not sure how Google/Firebase feels about that, though). The primary purpose is to reduce the friction needed for someone to try out the firebaserules api and to understand the firebaserules.

asciimike commented 6 years ago

I strongly recommend not making these keys public under any circumstances. Everyone is OK with sinking the cost of abuse until they get a $10k bill for mining bitcoins that aren't theirs ;)

I'd instead recommend building a simple website and using Google Consumer OAuth instead, or providing instructions for downloading and providing a service account. This is also sort of the point of the API explorer (though I understand the difference between the API explorer and the client library).

SamyPesse commented 6 years ago

I've just published expect-firestore, it brings an abstraction on top of the API to avoid writing functions mocks and instead use a dataset (the library generates the mocks automatically).

It has an API similar to targaryen.

@mcdonamp How can we test transactions (multiple set/update) ? A lot of our apps logic is build on top of transactions, and it doesn't seem testable yet.

@mcdonamp When will you open source / publish the protoc plugin ? We'd love to use it as soon as possible.

asciimike commented 6 years ago

Nice, thanks for doing that! We should bring back the "third party" section on our site to make these libs more discoverable.

Transaction support is hard, partly because I don't believe we yet support them in Rules (e.g. you can't write a rule that references data from different documents). We're working on the best design for this.

The protoc plugin is in open source review now, but unlikely to make it out before the new year due to the approvers being on vacation.

SamyPesse commented 6 years ago

@mcdonamp Any news on open sourcing the protoc plugin ?

But even if we could easily write rules, the impossibility to test batch sets is a deal breaker on our side (we have an important bunch of rules to ensure that some duplicated data are kept in sync and require a batch set).

I'm quite surprised, it didn't come out as a key feature of the rules backend when Firestore was designing. I'd be interested in learning how other company keep a consistent FB database for their apps without using and testing batch sets.

asciimike commented 6 years ago

@SamyPesse The protoc plugin launched today (https://github.com/firebase/protobuf-rules-gen). Would love feedback on it :)

Validating batch requests is coming shortly. Here's what we're planning:

// JS pseudo-code
var batch = firebase.batch();
batch.set('/posts/post6', {'text': 'foo bar baz...'});
batch.update('/users/myauthid', {'latestPost': 'post6'});
batch.commit();
// Rules
service firestore {
  match /posts/{postId} {
    allow write: if getAfter(/users/$(request.auth.uid)).data.latestPost == postId;
  }
}

Would love feedback on how this works, how you feel about the name getAfter(), etc.

SamyPesse commented 6 years ago

@mcdonamp Awesome! We'll give it a try (hopefully there will be a release to download instead of compiling from sources 😁).

I'm wondering if the getAfter should not be the default behaviour of get and instead we could have something similar to bolt's prior(): getPrior(/.../) or prior(get(/.../)).

I think the most common use case is to validate and operate on the "resulting" data, instead of the previous data.

But getAfter will work just fine for us :)

rockwotj commented 6 years ago

@SamyPesse this afternoon I'm going to create some binary releases.

Also some clarification about getAfter, it accesses the state of the database after the write - if it were to be allowed, it's the equivalent of newData in the Realtime Database (AKA this in bolt). The current get operator is the RealtimeDatabase equivalent to data (AKA prior() in bolt).

I do see that the bolt thing of making newData the default in writes and data the default in reads makes a lot of sense. However, it's a little hard to make that change now without breaking people.

rockwotj commented 6 years ago

I've created binaries for Linux & Mac here: https://github.com/firebase/protobuf-rules-gen/releases/tag/v0.1.0

I'll document how to use the binaries tomorrow morning, if you're unfamiliar with protoc plugins (which I'm guessing most people are; there is an example usage script if you want to see an example now).

If anyone needs Windows support please open an issue in that repo. Please give us feedback on that repo!

SamyPesse commented 6 years ago

@mcdonamp Where can we follow the progress on Firestore ? The releases notes don't seem updated: https://cloud.google.com/firestore/docs/release-notes

asciimike commented 6 years ago

@SamyPesse looks like Dan did an email blast to the google-cloud-firestore-discuss group with updates here. I'm also seeing some release notes on https://firebase.google.com/support/releases.

I've asked why the Cloud site doesn't have any, so hopefully we'll get an answer; otherwise, check the Firebase site for the latest updates.

(Also, sorry, have been on vacation most of the month, hence the delayed responses).

rockwotj commented 6 years ago

Update here, getAfter is finally launched, the release notes are here: https://firebase.google.com/support/releases

34r7h commented 5 years ago

Has there been any progress for a Firestore Bolt?

asciimike commented 5 years ago

No. What are the particular features you're interested in?