Add Instance level privileges/tokens

[Trovalo]

Proposal: I'd love to see instance-level permission management. This is what I mean by Instance-level

- Instance
  - Organizations
    - Buckets

As of now, It's not possible to access InfluxDB with a ReadOnly/WriteOnly token that is valid for multiple organizations, exception made for the "operator token", the equivalent of sysadmin, which should not be used for applications.

Current behavior: There is no way to create a token that allows access to more than one Organization

Desired behavior: Create and manage tokens that allow access to more than one Organization

Alternatives considered: I've tried with the v1 authorization (for backward compatibility), but the created user is still organization scoped. There is no way around it.

Use case: My personal use case is to have a single token with write-only permission on multiple Organizations, I have a centralized system, in which all the data pass through a single Telegraf gateway. Using InflxDB1 I've got no problems, as I just need a user with Write permission on all databases... by translating this into InfluxDB2, the best fit is to turn a DB into an Organization, but I can't have a token to access them all... which means the routing (made by the telegraf gateway) becomes hell. The whole idea started from a telegraf feature request https://github.com/influxdata/telegraf/issues/10778, you may find some more details about my use case there.

edit: typos

[lesam]

@Trovalo usually databases in V1 would convert into buckets, not organizations. Why do you want to have the data in separate organizations?

[Trovalo]

I'm aware that the "default" migration path should turn DB+RP into a bucket, in my case, the "Bucket per customer" and "Organization per customer" designs are both valid, the latest is cleaner but it suffers due to the auth token limitation.

As a reference, here is my current structure:

and this is the result of the "default" conversion (bucket per customer)

here is the other design (Org per customer):

Organization per customer design The org per customer design makes the query parametrization easier, as only the high-level connection (Organization) needs to change, and all the underlying structure is constant. On the practical side a query defined as SELECT ___ FROM standard.cpu or SELECT ___ FROM long.cpu will work on any Organization, only the connection string needs to be changed (to point to a different Org). (I know that's InfluxQL and not flux, it's just to give the idea)

bucket per customer design here the parametrization is more complex... as the bucket name is not fixed, therefore just changing the connection string will not suffice A query defined as SELECT ___ FROM customer1/standard.cpu or SELECT ___ FROM customer1/long.cpu won't work "for another customer" as the query code itself needs to be changed. Everything might work smoothly as long as you have only one bucket per customer because in that case, you can avoid it entirely and the query will use the default one, but if you have more than one bucket the name must change dynamically.

I'm using Grafana as the frontend tool, and it is not easy to make the bucket dynamic out of the box, the v2 connection is defined by: Org, Token, Default bucket.

[lesam]

@Trovalo the write API also requires the organization as a parameter - potentially you could select the token for the org at the same time you select the org? https://docs.influxdata.com/influxdb/v2.1/write-data/developer-tools/api/#send-a-write-request

Also how do your queries look in 1.x? 2.x supports exactly the same kind of InfluxQL queries via the compatibility API, so I don't think you should have changes there. If you're having flux issues with multiple buckets it would be helpful to give flux examples.

[Trovalo]

@lesam, by coding everything is possible, but not every tool allows you to do that, for example, if you write data via Telegraf the token is static, therefore you can't implement a get the right token logic, not nicely at least, it's doable by duplicating the output and filtering it properly, which leads to continuous maintenance/edits. For Telegraf the logic must be implemented in the config files, but the point is that the mapping Org-Token has to be managed somewhere, be it a table on a DB, or in the code itself, while I think this should not be the case, I expect this to be solved by the security part of the DBMS itself.

I'd normally expect to be able to create a security object (user/token) that has whatever permission is needed by the application, be it sysadmin ("operator token"), read-only on all, or just on a subset of objects (orgs, buckets) or any other possible combination.

Instance-level permission, or more generally permission management on the main system levels (Instance,DB,Objects for a SQL DB or Instance,DB,container for NoSQL DBs) is something I've always found in all DBMS and I'd like to see it also here as I think it will allow for higher flexibility.

[lesam]

@Trovalo if you write via telegraf the org would be static as well - I don't believe telegraf allows configuring the org separate from the token, or configuring the org dynamically based on incoming data.

[lesam]

See https://github.com/influxdata/telegraf/blob/master/plugins/outputs/influxdb_v2/README.md

[Trovalo]

@Trovalo if you write via telegraf the org would be static as well - I don't believe telegraf allows configuring the org separate from the token, or configuring the org dynamically based on incoming data.

I know that perfectly, that's exactly where my journey began... I saw there was https://github.com/influxdata/telegraf/issues/10778, proposed it and learned that it's pointless as there is no way to authenticate with permission on multiple organizations... So here I am... proposing a feature to expand permission/token management.

[lesam]

@samhld FYI

[spaghettidba]

I suppose that @Trovalo 's point can be summarized like this: A multi-tenant application with several Grafana dashboards was totally possible in InfluxDB v1 using a separate database for each tenant, without having to change the InfluxQL queries and using Telegraf to route the data to the correct database.

With v2, in the "one org for all" model, this is possible only with the compatibility endpoints (setting up mappings for database + RP = bucket), because in the v2 endpoints every dashboard would have to be changed to point to the correct buckets (it has to be static in flux).

The "one tenant - one org" model would make it even harder, because the current implementation of the write APIs and the security model (no user can access multiple orgs) make it impossible using a Telegraf gateway.

Changing the security model to allow access to multiple orgs would solve the problem in v2, both for "one org for all" and "one tenant - one org" models, without the need for compatibility endpoints.

[Trovalo]

Any update/consideration on this one? I know having instance level permission management might not be that easy as the system has been structured differently (with organization scoped security), but I'd like to know if this point has been considered/rejected/whatever

Thanks

[samhld]

@Trovalo we are considering ways to make this experience better in the coming months. I don't have a good answer for you yet but we will link to this issue on any work that starts!