Filter JSON before output

[adam-codeberg]

Quick question,

Is there any existing tooling to filter json views before output with the existing tooling?

Useage example; instances where you have a collection with items and those items have user ownership and you want to hide the items from non-owners when the collection is queried via a GET /api/collection

If so great, but if not I'll look into monkey patching the views.py functions in ramses or something.

[postatum]

Hi Adam.

You have two options:

Using nefertari-guards https://github.com/brandicted/nefertari-guards, which stores ACLs in database+es in a separate "_acl" field. Run your ramses app with auth=True and database_acls=True to use it (also you need to pip install it). Among its features is - when GETing collection, ACL filtering is performed (similar to what you described). E.g. If there are 2 items - item1 which "allow user1 view", and item2 which "allow user2 view" and user1 GETs collection, he will only see item1. Items are filtered by displaying those which allow access to any of user's principal identifiers and DO NOT deny access to any of user's principal identifiers.

Things to consider when using nefertari-guards are: a) ACL inheritance will probably be disabled in next versions. b) Collection ACL filtering doesn't respect order of ACEs in ACL. Thus if your ACL says "allow user1 view, deny everyone view" - in terms of Pyramid, this ACL allows view to user1, because it comes before Deny, but ACL filtering will still hide that item, because it denies access to one of user1's identifiers("everyone").

Using nefertari tools You could use "AfterIndex" nefertari event (https://nefertari.readthedocs.org/en/stable/event_handlers.html). Looks like "event.response" points to the response object which would be returned from view and it's probably what you need. I didn't test it, but it should work if you edit that response object directly.

[adam-codeberg]

Thanks for pointing me in the right direction and the insight into the next version. Will save me a lot of time trying to reinvent the wheel.

Greatly appreciate it.

[adam-codeberg]

The nefertari-guards mixin is working as expected for system users, however I'm stumped when it comes to defining _acl's for non system users.

Have tried inserting data via a range of means without luck.

Example;

POST :6543/v1/auth/register username='agent1' email='agent1@test.com' password='password' _acl="[{'action': 'allow', 'permission': 'all', 'principal': 'agent1'}]"
"(exceptions.ValueError) need more than 1 value to unpack"

PATCH :6543/v1/users/agent1 _acl="[{'action': 'allow', 'permission': 'all', 'principal': 'agent1'}]"
"(exceptions.ValueError) need more than 1 value to unpack"

POST :6543/v1/auth/register username='agent1' email='agent1@test.com' password='password' _acl="[(Allow, agent1, 'update')]" 
"(exceptions.ValueError) need more than 1 value to unpack"

PATCH :6543/v1/users/agent1 _acl="[(Allow, agent1, 'update')]" 
"(exceptions.ValueError) need more than 1 value to unpack"

Tried event handlers

@registry.add
def set_item_owner(event):
    """ Set owner of an item. """
    user = getattr(event.view.request, 'user', None)
    log.info('Set item owner to: {}'.format(str(user.username)))
    perms = [{"action": "allow", "permission": "all", "principal": str(user.username)}]
    if '_acl' not in event.fields and user is not None:
        event.set_field_value('_acl', perms)

Switched log.info to log.warning for debugging and discovered that the user auth model doesn’t seem to run assigned event handlers.

Cant get data into the _acl field for non system users. Must be overlooking something but not sure what.

Is this expected behaviour?

---

Edit: I've updated to the latest builds of ramses, nefertari, nefertari-guards. Having some partial success, will continue to run some tests and get back to you.

---

Ticket resolved, was able to fix the issue by updating to the latest release versions of ramses, nefertari, nefertari-guards

[postatum]

Ticket resolved, was able to fix the issue by updating to the latest release versions of ramses, nefertari, nefertari-guards

Good. I'll leave this ticket open for a while if you have more questions.

[adam-codeberg]

Things to consider when using nefertari-guards are: a) ACL inheritance will probably be disabled in next versions. b) Collection ACL filtering doesn't respect order of ACEs in ACL. Thus if your ACL says "allow user1 view, deny everyone view" - in terms of Pyramid, this ACL allows view to user1, because it comes before Deny, but ACL filtering will still hide that item, because it denies access to one of user1's identifiers("everyone").

Thanks for that. Re-reading helped solved a few inspected behaviours I was experiencing.

Items are filtered by displaying those which allow access to any of user's principal identifiers and DO NOT deny access to any of user's principal identifiers

What I'm trying to achieve is allow users to view the filtered collection summary, and to their own {items}, but Deny them from other users {items}

At the moment, it looks like {items} inherit the permissions from their collection, there for, since the collection filtering requires View permissions, all users have access to all other users {items}

I've had a look at before_show and after_show in hopes of being able to filter the {item} results from there, but no luck so far.

Is this issue scheduled to be resolved in the next update?

[jstoiko]

These 2 PRs have been merged into the respective develop branches of nefertari-guards and ramses and should address your issue. In essence, it disables acl inheritance when nefertari-guards is used. It makes sense because when using nefertari-guards, acls are stored at the object level and some objects may have permissive ACLs while their parent collections may deny things and we don't want to inherit from those deny rules in that case.

https://github.com/brandicted/ramses/pull/99 https://github.com/brandicted/nefertari-guards/pull/13

[adam-codeberg]

Sounds good. I will pull the development branch if I get a few spare moments this or next week.

Also permitting Allow to supersede Deny would be handy. For example;

Deny Everyone All
Allow Authenticated View,Create,Options
Allow g:admin Update,Remove

Where as in its current state with Denies superseding Allows you have to write many times more permissions to achieve the same effect.

Not sure why Pyramid does the opposite, but may be overlooking something http://docs.pylonsproject.org/projects/pyramid/en/latest/narr/security.html#creating-your-own-authentication-policy

[jstoiko]

Also permitting Allow to supersede Deny would be handy

in nefertari-guards? That's a limitation of ElasticSearch. The underlying ES query is using a combination of must and must_not filter. If an object had the ACEs in your example, nefertari-guards would query must_not deny everyone in combination with must allow authenticated... regardless of the order in which those ACEs are defined.