NetApp / harvest

Open-metrics endpoint for ONTAP and StorageGRID
https://netapp.github.io/harvest/latest
Apache License 2.0
141 stars 36 forks source link

Documentation - Least-privilege approach for REST role #2991

Closed mamoep closed 2 days ago

mamoep commented 1 month ago

Is your feature request related to a problem? Please describe. The documentation has a detailed list of permissions needed for legacy roles. For rest-role it is just a readonly for the whole api.

Describe the solution you'd like Please document a least-privilege approach for security login rest-role create command.

cgrinds commented 1 month ago

Hi @mamoep

"legacy" roles is a misnomer; perhaps a better name would be RBAC roles. Either way, the legacy roles are not going away, and both REST and ZAPI roles are defined in terms of those RBAC-based roles. In other words, when you modify a rest-role, ONTAP translates that REST role into an RBAC role and updates the RBAC access to satisfy the REST role. RBAC roles are the foundational mechanism to control access, and other higher-level ways of controlling access ultimately are implemented in terms of RBAC.

Harvest also uses ONTAP's REST private CLI passthrough. As of commit 5ae91b7d, there are 23 templates in the conf/rest directory that use an /api/private/ endpoint. Since the private CLI passthrough invokes the ONTAP CLI, those endpoints are controlled by RBAC permissions instead of REST permissions.

Here's an example:

Starting with a role that does not exist

security login rest-role show -role tmp-rest
There are no entries matching your query.

security login role show -role tmp-rest
There are no entries matching your query

Create a rest role granting access to /api/svm/svms

security login rest-role create -role tmp-rest -access readonly -api /api/svm/svms

Verify that the role was created with readonly access

security login rest-role show -role tmp-rest
               Role                                    Access
Vserver        Name            API                     Level
----------     -------------   -------------------     ------
umeng-aff300-01-02
               tmp-rest        /api/svm/svms           readonly

Creating the rest-role also caused an RBAC role to be created

Also notice that the RBAC command DEFAULT was modified too, with all access being removed.

security login role show -role tmp-rest
           Role          Command/                                      Access
Vserver    Name          Directory                               Query Level
---------- ------------- --------- ----------------------------------- --------
umeng-aff300-01-02
           tmp-rest      DEFAULT                                       none
                         template provision                            readonly
                         vserver                                       readonly
3 entries were displayed.

Since creating REST roles also sets the underlying RBAC permissions, we attempted several releases ago to unify the least privilege approach documentation by listing only the REST role permissions. If we had succeeded in doing that, it would have meant that setting the REST-roles would have enabled both a ZAPI and REST least privileged approach. Unfortunately, there were a few gaps that made us abandon that approach. For example, there was no REST role that would cause system health subsystem show to be added as an RBAC command. For instance, security login role show-rest -commands "system health subsystem show" returns nothing. While the REST collectors do not need that privilege, the ZAPI collectors do.

All of which is to say, we hear you and will add that documentation. 😄

Digging through my notes from the last time I looked at this, I had this list which may be incomplete:

security login rest-role create -role harvest-rest-role -access readonly -api /api/cluster
security login rest-role create -role harvest-rest-role -access readonly -api /api/cluster/counter/tables
security login rest-role create -role harvest-rest-role -access readonly -api /api/cluster/nodes
security login rest-role create -role harvest-rest-role -access readonly -api /api/network/ip/interfaces
security login rest-role create -role harvest-rest-role -access readonly -api /api/snapmirror/relationships
security login rest-role create -role harvest-rest-role -access readonly -api /api/storage/aggregates
security login rest-role create -role harvest-rest-role -access readonly -api /api/storage/disks
security login rest-role create -role harvest-rest-role -access readonly -api /api/storage/luns
security login rest-role create -role harvest-rest-role -access readonly -api /api/storage/qos/policies
security login rest-role create -role harvest-rest-role -access readonly -api /api/storage/qos/workloads
security login rest-role create -role harvest-rest-role -access readonly -api /api/storage/qos/workloads
security login rest-role create -role harvest-rest-role -access readonly -api /api/storage/shelves
security login rest-role create -role harvest-rest-role -access readonly -api /api/storage/volumes
security login rest-role create -role harvest-rest-role -access readonly -api /api/svm/svms
security login rest-role create -role harvest-rest-role -access readonly -api /api/security/ssh
mamoep commented 1 month ago

Thanks for elaborating. The decision to separate RBAC and REST role management in ONTAP was not a great idea.

I could find a match for your gap example: security login role show-rest -commands "system health" shows some output. It maps to /api/cluster/nodes. Maybe you can try again to fill the gaps.

cgrinds commented 1 month ago

Thanks @mamoep! What version of ONTAP are you running that showed output?

I'll update my text above to include this for searching in the future, but another important point is that Harvest makes extensive use of REST's private CLI passthrough support. As of 5ae91b7d, there are 23 templates in the conf/rest directory that use an /api/private/ endpoint. Since private CLI passthrough invokes the ONTAP CLI, those endpoints are gated on RBAC permissions instead of REST ones.

mamoep commented 1 month ago

I am running 9.13.P9 Is there a list of all the api/private calls available?

cgrinds commented 1 month ago

Here's the list as of commit 5ae91b7d0e978fc91c8f74f7f524a65c29b75fd1

grep -R 'api/private' conf/rest | sort | uniq
conf/rest/9.10.0/sensor.yaml:query:            api/private/cli/system/node/environment/sensors
conf/rest/9.10.0/svm.yaml:  - query: api/private/cli/vserver/cifs/server/security
conf/rest/9.10.0/svm.yaml:query:                    api/private/cli/vserver
conf/rest/9.12.0/disk.yaml:  - query: api/private/cli/disk
conf/rest/9.12.0/fru.yaml:query:                        api/private/cli/system/fru-check
conf/rest/9.12.0/lif.yaml:  - query: api/private/cli/network/interface
conf/rest/9.12.0/netconnections.yaml:query:                        api/private/cli/network/connections/active
conf/rest/9.12.0/netport.yaml:  - query: api/private/cli/network/port
conf/rest/9.12.0/netport.yaml:  - query: api/private/cli/network/port/ifgrp
conf/rest/9.12.0/node.yaml:  - query: api/private/cli/node
conf/rest/9.12.0/qos_policy_adaptive.yaml:query:              api/private/cli/qos/adaptive-policy-group
conf/rest/9.12.0/qos_policy_fixed.yaml:query:              api/private/cli/qos/policy-group
conf/rest/9.12.0/qos_workload.yaml:query:            api/private/cli/qos/workload
conf/rest/9.12.0/qtree.yaml:  - query: api/private/cli/qtree
conf/rest/9.12.0/shelf.yaml:  - query: api/private/cli/storage/shelf
conf/rest/9.12.0/snapmirror.yaml:query:                        api/private/cli/snapmirror
conf/rest/9.12.0/snapshot.yaml:query:                    api/private/cli/snapshot
conf/rest/9.12.0/snapshotpolicy.yaml:query:                    api/private/cli/snapshot/policy
conf/rest/9.12.0/subsystem.yaml:query:      api/private/cli/system/health/subsystem
conf/rest/9.12.0/volume.yaml:  - query: api/private/cli/volume/efficiency/stat
conf/rest/9.12.0/volume.yaml:  - query: api/private/cli/volume/footprint
conf/rest/9.12.0/volume.yaml:  - query: api/private/cli/volume/snapshot/autodelete
conf/rest/9.12.0/volume.yaml:query:                    api/private/cli/volume
conf/rest/9.6.0/cifs_share.yaml:query:              api/private/cli/vserver/cifs/share
conf/rest/9.7.0/ontap_s3.yaml:  - query: api/private/cli/vserver/object-store-server/bucket
conf/rest/9.7.0/ontap_s3_policy.yaml:query:            api/private/cli/vserver/object-store-server/bucket/policy
conf/rest/9.8.0/exports.yaml:query:                    api/private/cli/export-policy/rule
conf/rest/9.9.0/svm.yaml:  - query: api/private/cli/vserver/cifs/server/security
conf/rest/9.9.0/svm.yaml:query:                    api/private/cli/vserver
cgrinds commented 1 month ago

I checked a 9.15.1 cluster and the problem I mentioned above still exists. That is, security login role show-rest -commands "system health subsystem show" returns "There are no entries matching your query."

security login role show-rest -commands "system health" map to /api/cluster/nodes. /api/cluster/nodes is one of the permissions included in the list above.

Unfortunately, adding /api/cluster/nodes does not enable the RBAC permission system health subsystem show which is required by the ZAPI Subsystem collector and the Rest Subsystem collector.

mamoep commented 4 weeks ago

The RBAC permission system health is added, which includes the system health subsystem show. This is not least privilege possible but maybe with some compromise, we can get all that is needed.

mamoep commented 4 weeks ago

With the following role I could get all of the listed private endpoints with only 2 exceptions. network connections active - The template is disabled by default system fru-check - I could only find a template in discussions and the endpoint mentionend in documentation, but no template in the code base.

security login rest-role create -role harvest-rest-role -access readonly -api /api/cloud/targets
security login rest-role create -role harvest-rest-role -access readonly -api /api/cluster
security login rest-role create -role harvest-rest-role -access readonly -api /api/cluster/counter/tables
security login rest-role create -role harvest-rest-role -access readonly -api /api/cluster/mediators
security login rest-role create -role harvest-rest-role -access readonly -api /api/cluster/metrocluster/diagnostics
security login rest-role create -role harvest-rest-role -access readonly -api /api/cluster/nodes
security login rest-role create -role harvest-rest-role -access readonly -api /api/cluster/ntp/servers
security login rest-role create -role harvest-rest-role -access readonly -api /api/cluster/peers
security login rest-role create -role harvest-rest-role -access readonly -api /api/cluster/sensors
security login rest-role create -role harvest-rest-role -access readonly -api /api/name-services/ldap
security login rest-role create -role harvest-rest-role -access readonly -api /api/name-services/nis
security login rest-role create -role harvest-rest-role -access readonly -api /api/network/ethernet/ports
security login rest-role create -role harvest-rest-role -access readonly -api /api/network/fc/ports
security login rest-role create -role harvest-rest-role -access readonly -api /api/network/ip/interfaces
security login rest-role create -role harvest-rest-role -access readonly -api /api/network/ip/ports
security login rest-role create -role harvest-rest-role -access readonly -api /api/network/ip/routes
security login rest-role create -role harvest-rest-role -access readonly -api /api/private/cli
security login rest-role create -role harvest-rest-role -access readonly -api /api/private/support/alerts
security login rest-role create -role harvest-rest-role -access readonly -api /api/protocols/cifs/services
security login rest-role create -role harvest-rest-role -access readonly -api /api/protocols/cifs/sessions
security login rest-role create -role harvest-rest-role -access readonly -api /api/protocols/cifs/shares
security login rest-role create -role harvest-rest-role -access readonly -api /api/protocols/locks
security login rest-role create -role harvest-rest-role -access readonly -api /api/protocols/ndmp/sessions
security login rest-role create -role harvest-rest-role -access readonly -api /api/protocols/nfs/connected-clients
security login rest-role create -role harvest-rest-role -access readonly -api /api/protocols/nfs/export-policies
security login rest-role create -role harvest-rest-role -access readonly -api /api/protocols/s3/buckets
security login rest-role create -role harvest-rest-role -access readonly -api /api/protocols/s3/services
#s3 is buggy in 9.15, can use protocols endpoint instead
#security login rest-role create -role harvest-rest-role -access readonly -api /api/protocols
security login rest-role create -role harvest-rest-role -access readonly -api /api/security
security login rest-role create -role harvest-rest-role -access readonly -api /api/security/accounts
security login rest-role create -role harvest-rest-role -access readonly -api /api/security/audit/destinations
security login rest-role create -role harvest-rest-role -access readonly -api /api/security/certificates
security login rest-role create -role harvest-rest-role -access readonly -api /api/security/login/messages
security login rest-role create -role harvest-rest-role -access readonly -api /api/security/ssh
security login rest-role create -role harvest-rest-role -access readonly -api /api/snapmirror/relationships
security login rest-role create -role harvest-rest-role -access readonly -api /api/storage/aggregates
security login rest-role create -role harvest-rest-role -access readonly -api /api/storage/disks
security login rest-role create -role harvest-rest-role -access readonly -api /api/storage/flexcache/flexcaches
security login rest-role create -role harvest-rest-role -access readonly -api /api/storage/luns
security login rest-role create -role harvest-rest-role -access readonly -api /api/storage/namespaces
security login rest-role create -role harvest-rest-role -access readonly -api /api/storage/qtrees
security login rest-role create -role harvest-rest-role -access readonly -api /api/storage/qos/policies
security login rest-role create -role harvest-rest-role -access readonly -api /api/storage/qos/workloads
security login rest-role create -role harvest-rest-role -access readonly -api /api/storage/quota/reports
security login rest-role create -role harvest-rest-role -access readonly -api /api/storage/shelves
security login rest-role create -role harvest-rest-role -access readonly -api /api/storage/volumes
security login rest-role create -role harvest-rest-role -access readonly -api /api/support/auto-update
security login rest-role create -role harvest-rest-role -access readonly -api /api/support/autosupport
security login rest-role create -role harvest-rest-role -access readonly -api /api/support/ems/destinations
security login rest-role create -role harvest-rest-role -access readonly -api /api/support/ems/events
security login rest-role create -role harvest-rest-role -access readonly -api /api/svm/peers
security login rest-role create -role harvest-rest-role -access readonly -api /api/svm/svms
cgrinds commented 4 weeks ago

Thanks @mamoep!

There are 20 permissions errors (see below) when running Harvest with Rest and RestPerf collectors when using the permission list from above.

Three of the "permission denied" failures are from public APIs and the other 17 are from private CLIs. The three public APIs that failed are:

Error: command failed: Specified URI path is invalid or not supported. Verify that the URI contains only valid characters.


- /api/storage/qtrees
This one works fine

- /api/support/autosupport 
This one is in your list, was added on this cluster, but still returns a 403. Bug?

```bash
rg 'Permission denied' /tmp/n.t | rg -o 'API:.* Poller' | cut -d' ' -f2 | cut -d'?' -f1 | sort | uniq
/api/private/cli/aggr
/api/private/cli/cluster/date
/api/private/cli/disk
/api/private/cli/network/interface
/api/private/cli/node
/api/private/cli/qos/adaptive-policy-group
/api/private/cli/qos/policy-group
/api/private/cli/qos/workload
/api/private/cli/snapmirror
/api/private/cli/snapshot/policy
/api/private/cli/storage/failover
/api/private/cli/storage/shelf
/api/private/cli/system/chassis/fru
/api/private/cli/system/health/subsystem
/api/private/cli/volume
/api/private/cli/vserver
/api/private/cli/vserver/object-store-server/bucket/policy
/api/protocols/s3/buckets
/api/storage/qtrees
/api/support/autosupport

Creating a new role (h-tmp-role) with your list + the one missing qtree role results in security login rest-role show -role h-tmp-role displaying 42 entries and security login role show -role h-tmp-role displays 167 entries.

mamoep commented 4 weeks ago

I think I forgot the most basic for private cli. :) security login rest-role create -role harvest-rest-role -access readonly -api /api/private/cli

s3 is a known issue: https://mysupport.netapp.com/site/bugs-online/product/ONTAP/JiraNgage/CONTAP-210232

I added the missing parts to my initial list

cgrinds commented 4 weeks ago

Do you still consider it "least privilege" if you're required to add /api/private/cli?

Adding /api/private/cli is identical to running security login role create -role harvest2-role -access readonly -cmddirname "DEFAULT", which is adding readonly access for all CLI apis.

Left side is before adding /api/private/cli and right side is after

image
mamoep commented 4 weeks ago

Guess your right, that defeats the purpose. :(

mamoep commented 4 weeks ago

Looks different in my simulator. Which ONTAP version are you running?

sim01::*> security login role show -vserver sim01 -role harvest-rest-role
           Role          Command/                                      Access
Vserver    Name          Directory                               Query Level
---------- ------------- --------- ----------------------------------- --------
sim01 harvest-rest-role
                         DEFAULT                                       none
                         system node run                               readonly
                         system node systemshell                       readonly

sim01::*> security login rest-role show -vserver sim01 -role harvest-rest-role
               Role                                    Access
Vserver        Name            API                     Level
----------     -------------   -------------------     ------
sim01     harvest-rest-role
                               /api/private/cli        readonly
cgrinds commented 4 weeks ago
version
NetApp Release Mightysquirrel__9.15.1: Fri May 24 05:03:10 UTC 2024
mamoep commented 4 weeks ago

So mapping inconsistencies between different ONTAP versions. This whole REST thing is a mess... Do at least the "Permission denied" errors disappear when adding the private cli endpoint?

cgrinds commented 4 weeks ago

Perhaps what you are seeing is a simulator issue. I'll try on another cluster and report back.

If I add /api/private/cli (which adds the wildcard RBAC), more things work. Here are the other misses:

/api/name-services/ldap
/api/name-services/nis
/api/storage/quota/reports
/api/svm/peers

This API, used by the Rest:Health plugin, still fails which is confusing /api/private/support/alerts

cgrinds commented 4 weeks ago

Adding /api/private/cli on a NetApp Release 9.13.1P3: Wed Sep 20 01:37:11 UTC 2023 cluster does NOT touch the DEFAULT command.

Adding /api/private/cli on a NetApp Release 9.14.1P2: Tue Mar 19 23:28:08 UTC 2024 cluster changes the DEFAULT command from none to readonly.

mamoep commented 4 weeks ago

/api/private/support/alerts is not equal /api/private/cli. But it can be added to the rest-role. I guess they hide some stuff in there. :)

Can you talk to some ONTAP people about the DEFAULT command getting added in 9.15?

cgrinds commented 3 weeks ago

hi @mamoep I heard back from ONTAP. This change was intentional and not a bug (burt 1496317). When you specify the endpoint /api/private/cli as readonly, the DEFAULT access for the traditional role will also change to readonly.

I wonder if the behavior you want could be achieved by not adding /api/private/cli and instead adding each child tree individually /api/private/cli/aggr, /api/private/cli/cluster/date, /api/private/cli/disk, etc?

cgrinds commented 3 weeks ago

I have confirmed that if you add a subpath of api/private/cli the DEFAULT command access remains none.

security login role show h-tmp-role
           Role          Command/                                      Access
Vserver    Name          Directory                               Query Level
---------- ------------- --------- ----------------------------------- --------
umeng-aff300-01-02
           h-tmp-role    DEFAULT                                       none
                         snapmirror object-store config                readonly
                         storage aggregate object-store config         readonly
3 entries were displayed.

security login rest-role create -role h-tmp-role -access readonly -api /api/private/cli/system/chassis/fru

umeng-aff300-01-02::> security login role show h-tmp-role
           Role          Command/                                      Access
Vserver    Name          Directory                               Query Level
---------- ------------- --------- ----------------------------------- --------
umeng-aff300-01-02
           h-tmp-role    DEFAULT                                       none
                         snapmirror object-store config                readonly
                         storage aggregate object-store config         readonly
                         system chassis fru                            readonly
4 entries were displayed.
mamoep commented 3 weeks ago

Subpath doesn't work on 9.13.1

security login rest-role create -role harvest-rest-role -api /api/private/cli/system/chassis/fru -access readonly

Error: command failed: URI does not exist.

The BURT you mentioned is not public readable. I speculate global readonly and subpath availability go hand in hand. Could you confirm this with the ONTAP team? Will this change be back-ported to 9.13 or 9.14?

cgrinds commented 2 days ago

3047 updates documentation and closes issue.

@mamoep I'm waiting to hear back from ONTAP on whether this change can be back-ported and will update this issue when I have an answer.