Per-document access control

[benliddicott]

Yes, I know... See also COUCHDB-615 on https://issues.apache.org/jira/browse/COUCHDB-615

0. Summary

Convention + Filtered views + intersections = access control.

Start with a standardised-by-convention honour-system access control scheme, which can be implemented client-side as an honour system, or enforced by a proxy. Add a system view which gives the sd for each document.

Add filtered views to (COUCHDB-707) provide server-side support. Then add intersections (joins) with other filtered views so access control information is indexed.

Finally add ability to optionally apply access control filters transparently to all views.

1. Honour System access control.

Extend the _security document to name additional groups than admin and member. Specifically add "reader" (implicit read of all non-system documents) and "restricted" (no implicit access), and allow adding any number of arbitrary groups.

Extend the userCtx to contain the property "principals" being a list of DB groups the user is in. The user and the roles generate the list of principals at request time. Therefore only principals need to be consulted for access control.

e.g. the

{
  "db":"mailbox",
  "name":"bob",
  "roles":[
    "cartographers"
    ],
    "principals":[
      "user:bob",
      "role:cartographers",
      "group:project X Admins"
      "db:non-admin",
      "db:restricted"
    ]
}

Each document can have a new system property, _sd. This can be consulted to discover desired access. An absent _sd means anyone may access. An empty _sd means only admins. Otherwise as described by the sd. Levels are read-only, and full access (including delete).

_sd:{ "user:bob":"rw", "role:cartographers":"r", "group:project X Admins":"rw" }

We now have a client-side honour system access control.

2. Allow filter functions to be used for views and for document reads.

This would allow users to specify a filter function which implemented access control to see only parts of the view they wish to see.

The filter function would be applied before the reduce step. Queries without a filter function would proceed as at present. Queries with a filter function will

On the honour system, this would be slow for views because it requires reading the document to retrieve the sd, unless the sd were output in the value.

Need to ensure multiple filter functions can be used together.

3. Allow intersection queries when querying views. (Yes, a bit like a join.)

Specify multiple views (potentially each with their own filter).

The output of the first view is the only one returned, but it is filtered by only accepting the document IDs which are produced by the other views.

4. Security Descriptor view

Create a new system view _doc_sd which emits an entry for each document _sd entry. If the _sd is empty, it will emit the implicit equivalents "documentID","db:readers","r"

5. Now we can (if configured) implement access control by implicitly querying _doc_sd

for intersection with the usrCtx.groups as keys.

6. Write access control

This can, as now, be implemented by the validation function.

7. Protection of namespaces from pollution by downlevel users

This can be provided by validation functions. These would consult a _namespace_sd document which would contain prefix to _sd mappings.

[benliddicott]

To be clear, if this issue is accepted, I would propose breaking it separate issues for each step, starting with "Honour system access control" then adding "filtered views", and "implicit filters".

[nolanlawson]

I think this is a feature that's badly needed for CouchDB, but I would prefer to have it solved in CouchDB first before getting solved in PouchDB Server. The goal of PouchDB Server is to basically just provide a direct implementation of CouchDB; to implement such a big feature in PouchDB Server before it exists in CouchDB would threaten to bifurcate the implementations, which we'd like to avoid.

(See also the recent discussion around _bulk_get, which took about a year (!) of discussion/implementation to get in, and was implemented simultaneously in PouchDB Server and CouchDB. You may want to try moving your proposal to the CouchDB dev mailing list or their Jira to see what their feedback is.)

[benliddicott]

@nolanlawson , the issue is on their Jira since Jan 2010. I have copied this issue there too. crickets

IMO this should be implemented as an optional add-on, but it does first need a small amount of support from the official project to allow new system properties to hold the security information (_sd, roles, etc).

[daleharvey]

There has been more discussion about an alternative implementation recently @ http://mail-archives.apache.org/mod_mbox/incubator-couchdb-dev/201509.mbox/%3CCAHdjipL43uf2P3vNS5UbWs2GpB0cAM9vM_HgG5P3UGgGrmVEDw@mail.gmail.com%3E

I am running into the need for per document access control as well, but I really do not like the design_doc based declerative approach that CouchDB takes, currently I am intercepting http requests on the server before they hit the database and responding appropriately, I opened an issue @ https://github.com/pouchdb/pouchdb/issues/4369 to start discussing it but havent kicked it off yet. the http implementation works well but currently requires knowledge of all the code paths that can be involved in a read or a write so I would like to add some hooks to make it more generically understandable.

I think either way this isnt a thing that is going to land in pouchdb / server / couchdb any time soon in a way that a lot of people are going to like, so I would like to make it as easy as possible for people to put it on a layer about pouchdb (but below the server) so people can build their own implementations easily and maybe one will win out

[yaronyg]

We will eventually have to have this for Thali as well. Although for our immediate needs it turns out we can sneak by for a little while with per DB ACLs.

[cdaringe]

If I already have ACL controls in place, auth in place (app rest API+Postgres), and pouch>couch proxy in place, @daleharvey, is it your opinion that these doc level/db level access authorizations occur in the app-level API layer? This is what I've been leaning towards vs spreading perms into various systems (eg into couch)

[marten-de-vries]

rcouch already has/had an implementation of this that's relatively easy to implement: validate_doc_read (https://github.com/rcouch/rcouch/wiki/Validate-documents-on-read). Yes, it's design doc based, and while I'd like to see the embedding approach @daleharvey talked about be made easier, design docs aren't going away anytime soon. I think we could profit from having both (even if it's just because you don't have to code your own server but can just spin up pouchdb-server).

If CouchDB gives JS rewrites a 'likely to be included at some point' status, that could be an option too. Shouldn't be too hard either. Both is just a few changes to pouchdb-rewrite/pouchdb-validation.

[nolanlawson]

This has been discussed at the CouchDB Dev Summit and hopefully we will have a more concrete plan soon

[orolle]

Any knowledge when per-document access control is planed for CouchDB? A quick google search did not provide any answer.