Easier to set per user and per key permissions for Key Value stores and object stores.

[bjorndm]

Proposed change

I propose that it should be easier to set per user and per key permissions for Key Value stores and object stores.

While it is possible to limit access to a key value store using a configuration like this: https://github.com/nats-io/nats-server/issues/3548, this has several downsides:

1. The configuration grows large if there are many keys to grant access to, this requires 1 line of configuration per bucket, and two lines of configuration per key.
2. It seem it is not possible to list or watch only certain allowed keys in a bucket. Now the list/wacth api uses "$JS.API.CONSUMER.CREATE.KV_bucket.*.$KV.bucket.>" as the API topic. Changing > to the key doesn't work.
3. It makes it hard to use the k/v storage and object storage with many users.

Use case

For my job, I want to use the key/value store and object store of NATS to store sensitive private medical data, with a per-key permission per user for each key in both stores. There will be many users each with many records. Certain users have the right to see the medical files of other users. While it could perhaps be possible now, it seems to be difficult, especially because there seems to be no way to watch or list a bucket and only get the keys the user has access to.

Contribution

With this feature I am more likely to get my employer rent a managed platform on Synadia. It is not likely that they will allow me to work on this feature though.

[derekcollison]

We agree, stay tuned. We have plans for Synadia Control Plane to allow easier setup of permissions for KVs and ObjectStores.

[bjorndm]

That's nice, but since I'm using NATS callout authentication, because each patient would be mapped to a NATS user using various existing authentication methods. It would be great if there was also some improvement on the API side.

Also I looked at the code but jsStreamInfoRequest only filters based on the API request, not on any permissions. This is not convenient because for privacy goals, it should be as if the keys/streams which a user is not allowed to access do not exist at all.

[derekcollison]

There are ways to do this with auth callouts.

Also why would a user be given access to stream info at all?

[bjorndm]

Yes, and I already implemented a prototype. But the API is somewhat difficult to use, I have to add several of API-specific permissions on the stream level, such as for direct get, direct set, ... It would be easier if we could specify it on the key/value and key level.

If I am not mistaken, jsStreamInfoRequest is called on the client jetstream.KeyValue.Watch API which is needed to iterate over the files of a user. One user might have several files they have access to in a bucket. For example, health care professional may be allowed to see the files of several patients. The HCP has to be able to iterate over the keys of these files.

[derekcollison]

The server primitives need to be supported, but we are looking at higher level primitives that might be able to help at some point in the future. You might also consider partnering with Synadia on a design, this works very well with our customers, but totally up to you.

Looping in @bruth from our side who is familiar with the healthcare space.

[bjorndm]

Well, yes, the low level primitives can stay how they are, but some way to generate them, like with a jwt.KeyValuePermissions struct in Go would be nice. I would like to work with Synadia but I probably need some working prototype before I can get some budget.

For jsStreamInfoRequest, a new API could be added that does filter based on the user's permissions, jsStreamInfoRequestForUser or such.

[Zetanova]

As an idea, this could be implemented with some new entity like virtual-bucket or scoped-bucket that have properties like a FilterSubjects attach to them.

my_bucket: dev.101.name: device101 dev.102.name: device102

dev101_bucket (scoped-bucket): Bucket: my_bucket FilterSubjects: dev.101, dev.101.> MaxAge: 60d //for GDPR

nats kv watch dev101_bucket