Closed mamoep closed 2 days ago
Hi @mamoep
"legacy" roles is a misnomer; perhaps a better name would be RBAC roles. Either way, the legacy roles are not going away, and both REST and ZAPI roles are defined in terms of those RBAC-based roles. In other words, when you modify a rest-role, ONTAP translates that REST role into an RBAC role and updates the RBAC access to satisfy the REST role. RBAC roles are the foundational mechanism to control access, and other higher-level ways of controlling access ultimately are implemented in terms of RBAC.
Harvest also uses ONTAP's REST private CLI passthrough. As of commit 5ae91b7d, there are 23 templates in the conf/rest
directory that use an /api/private/
endpoint. Since the private CLI passthrough invokes the ONTAP CLI, those endpoints are controlled by RBAC permissions instead of REST permissions.
Here's an example:
security login rest-role show -role tmp-rest
There are no entries matching your query.
security login role show -role tmp-rest
There are no entries matching your query
security login rest-role create -role tmp-rest -access readonly -api /api/svm/svms
security login rest-role show -role tmp-rest
Role Access
Vserver Name API Level
---------- ------------- ------------------- ------
umeng-aff300-01-02
tmp-rest /api/svm/svms readonly
Also notice that the RBAC command DEFAULT
was modified too, with all access being removed.
security login role show -role tmp-rest
Role Command/ Access
Vserver Name Directory Query Level
---------- ------------- --------- ----------------------------------- --------
umeng-aff300-01-02
tmp-rest DEFAULT none
template provision readonly
vserver readonly
3 entries were displayed.
Since creating REST roles also sets the underlying RBAC permissions, we attempted several releases ago to unify the least privilege approach documentation by listing only the REST role permissions. If we had succeeded in doing that, it would have meant that setting the REST-roles would have enabled both a ZAPI and REST least privileged approach. Unfortunately, there were a few gaps that made us abandon that approach. For example, there was no REST role that would cause system health subsystem show
to be added as an RBAC command. For instance, security login role show-rest -commands "system health subsystem show"
returns nothing. While the REST collectors do not need that privilege, the ZAPI collectors do.
All of which is to say, we hear you and will add that documentation. 😄
Digging through my notes from the last time I looked at this, I had this list which may be incomplete:
security login rest-role create -role harvest-rest-role -access readonly -api /api/cluster
security login rest-role create -role harvest-rest-role -access readonly -api /api/cluster/counter/tables
security login rest-role create -role harvest-rest-role -access readonly -api /api/cluster/nodes
security login rest-role create -role harvest-rest-role -access readonly -api /api/network/ip/interfaces
security login rest-role create -role harvest-rest-role -access readonly -api /api/snapmirror/relationships
security login rest-role create -role harvest-rest-role -access readonly -api /api/storage/aggregates
security login rest-role create -role harvest-rest-role -access readonly -api /api/storage/disks
security login rest-role create -role harvest-rest-role -access readonly -api /api/storage/luns
security login rest-role create -role harvest-rest-role -access readonly -api /api/storage/qos/policies
security login rest-role create -role harvest-rest-role -access readonly -api /api/storage/qos/workloads
security login rest-role create -role harvest-rest-role -access readonly -api /api/storage/qos/workloads
security login rest-role create -role harvest-rest-role -access readonly -api /api/storage/shelves
security login rest-role create -role harvest-rest-role -access readonly -api /api/storage/volumes
security login rest-role create -role harvest-rest-role -access readonly -api /api/svm/svms
security login rest-role create -role harvest-rest-role -access readonly -api /api/security/ssh
Thanks for elaborating. The decision to separate RBAC and REST role management in ONTAP was not a great idea.
I could find a match for your gap example:
security login role show-rest -commands "system health"
shows some output. It maps to /api/cluster/nodes
. Maybe you can try again to fill the gaps.
Thanks @mamoep! What version of ONTAP are you running that showed output?
I'll update my text above to include this for searching in the future, but another important point is that Harvest makes extensive use of REST's private CLI passthrough support. As of 5ae91b7d, there are 23 templates in the conf/rest
directory that use an /api/private/
endpoint. Since private CLI passthrough invokes the ONTAP CLI, those endpoints are gated on RBAC permissions instead of REST ones.
I am running 9.13.P9
Is there a list of all the api/private
calls available?
Here's the list as of commit 5ae91b7d0e978fc91c8f74f7f524a65c29b75fd1
grep -R 'api/private' conf/rest | sort | uniq
conf/rest/9.10.0/sensor.yaml:query: api/private/cli/system/node/environment/sensors
conf/rest/9.10.0/svm.yaml: - query: api/private/cli/vserver/cifs/server/security
conf/rest/9.10.0/svm.yaml:query: api/private/cli/vserver
conf/rest/9.12.0/disk.yaml: - query: api/private/cli/disk
conf/rest/9.12.0/fru.yaml:query: api/private/cli/system/fru-check
conf/rest/9.12.0/lif.yaml: - query: api/private/cli/network/interface
conf/rest/9.12.0/netconnections.yaml:query: api/private/cli/network/connections/active
conf/rest/9.12.0/netport.yaml: - query: api/private/cli/network/port
conf/rest/9.12.0/netport.yaml: - query: api/private/cli/network/port/ifgrp
conf/rest/9.12.0/node.yaml: - query: api/private/cli/node
conf/rest/9.12.0/qos_policy_adaptive.yaml:query: api/private/cli/qos/adaptive-policy-group
conf/rest/9.12.0/qos_policy_fixed.yaml:query: api/private/cli/qos/policy-group
conf/rest/9.12.0/qos_workload.yaml:query: api/private/cli/qos/workload
conf/rest/9.12.0/qtree.yaml: - query: api/private/cli/qtree
conf/rest/9.12.0/shelf.yaml: - query: api/private/cli/storage/shelf
conf/rest/9.12.0/snapmirror.yaml:query: api/private/cli/snapmirror
conf/rest/9.12.0/snapshot.yaml:query: api/private/cli/snapshot
conf/rest/9.12.0/snapshotpolicy.yaml:query: api/private/cli/snapshot/policy
conf/rest/9.12.0/subsystem.yaml:query: api/private/cli/system/health/subsystem
conf/rest/9.12.0/volume.yaml: - query: api/private/cli/volume/efficiency/stat
conf/rest/9.12.0/volume.yaml: - query: api/private/cli/volume/footprint
conf/rest/9.12.0/volume.yaml: - query: api/private/cli/volume/snapshot/autodelete
conf/rest/9.12.0/volume.yaml:query: api/private/cli/volume
conf/rest/9.6.0/cifs_share.yaml:query: api/private/cli/vserver/cifs/share
conf/rest/9.7.0/ontap_s3.yaml: - query: api/private/cli/vserver/object-store-server/bucket
conf/rest/9.7.0/ontap_s3_policy.yaml:query: api/private/cli/vserver/object-store-server/bucket/policy
conf/rest/9.8.0/exports.yaml:query: api/private/cli/export-policy/rule
conf/rest/9.9.0/svm.yaml: - query: api/private/cli/vserver/cifs/server/security
conf/rest/9.9.0/svm.yaml:query: api/private/cli/vserver
I checked a 9.15.1 cluster and the problem I mentioned above still exists. That is, security login role show-rest -commands "system health subsystem show"
returns "There are no entries matching your query."
security login role show-rest -commands "system health"
map to /api/cluster/nodes
. /api/cluster/nodes
is one of the permissions included in the list above.
Unfortunately, adding /api/cluster/nodes
does not enable the RBAC permission system health subsystem show
which is required by the ZAPI Subsystem
collector and the Rest Subsystem
collector.
The RBAC permission system health
is added, which includes the system health subsystem show
. This is not least privilege possible but maybe with some compromise, we can get all that is needed.
With the following role I could get all of the listed private endpoints with only 2 exceptions.
network connections active
- The template is disabled by default
system fru-check
- I could only find a template in discussions and the endpoint mentionend in documentation, but no template in the code base.
security login rest-role create -role harvest-rest-role -access readonly -api /api/cloud/targets
security login rest-role create -role harvest-rest-role -access readonly -api /api/cluster
security login rest-role create -role harvest-rest-role -access readonly -api /api/cluster/counter/tables
security login rest-role create -role harvest-rest-role -access readonly -api /api/cluster/mediators
security login rest-role create -role harvest-rest-role -access readonly -api /api/cluster/metrocluster/diagnostics
security login rest-role create -role harvest-rest-role -access readonly -api /api/cluster/nodes
security login rest-role create -role harvest-rest-role -access readonly -api /api/cluster/ntp/servers
security login rest-role create -role harvest-rest-role -access readonly -api /api/cluster/peers
security login rest-role create -role harvest-rest-role -access readonly -api /api/cluster/sensors
security login rest-role create -role harvest-rest-role -access readonly -api /api/name-services/ldap
security login rest-role create -role harvest-rest-role -access readonly -api /api/name-services/nis
security login rest-role create -role harvest-rest-role -access readonly -api /api/network/ethernet/ports
security login rest-role create -role harvest-rest-role -access readonly -api /api/network/fc/ports
security login rest-role create -role harvest-rest-role -access readonly -api /api/network/ip/interfaces
security login rest-role create -role harvest-rest-role -access readonly -api /api/network/ip/ports
security login rest-role create -role harvest-rest-role -access readonly -api /api/network/ip/routes
security login rest-role create -role harvest-rest-role -access readonly -api /api/private/cli
security login rest-role create -role harvest-rest-role -access readonly -api /api/private/support/alerts
security login rest-role create -role harvest-rest-role -access readonly -api /api/protocols/cifs/services
security login rest-role create -role harvest-rest-role -access readonly -api /api/protocols/cifs/sessions
security login rest-role create -role harvest-rest-role -access readonly -api /api/protocols/cifs/shares
security login rest-role create -role harvest-rest-role -access readonly -api /api/protocols/locks
security login rest-role create -role harvest-rest-role -access readonly -api /api/protocols/ndmp/sessions
security login rest-role create -role harvest-rest-role -access readonly -api /api/protocols/nfs/connected-clients
security login rest-role create -role harvest-rest-role -access readonly -api /api/protocols/nfs/export-policies
security login rest-role create -role harvest-rest-role -access readonly -api /api/protocols/s3/buckets
security login rest-role create -role harvest-rest-role -access readonly -api /api/protocols/s3/services
#s3 is buggy in 9.15, can use protocols endpoint instead
#security login rest-role create -role harvest-rest-role -access readonly -api /api/protocols
security login rest-role create -role harvest-rest-role -access readonly -api /api/security
security login rest-role create -role harvest-rest-role -access readonly -api /api/security/accounts
security login rest-role create -role harvest-rest-role -access readonly -api /api/security/audit/destinations
security login rest-role create -role harvest-rest-role -access readonly -api /api/security/certificates
security login rest-role create -role harvest-rest-role -access readonly -api /api/security/login/messages
security login rest-role create -role harvest-rest-role -access readonly -api /api/security/ssh
security login rest-role create -role harvest-rest-role -access readonly -api /api/snapmirror/relationships
security login rest-role create -role harvest-rest-role -access readonly -api /api/storage/aggregates
security login rest-role create -role harvest-rest-role -access readonly -api /api/storage/disks
security login rest-role create -role harvest-rest-role -access readonly -api /api/storage/flexcache/flexcaches
security login rest-role create -role harvest-rest-role -access readonly -api /api/storage/luns
security login rest-role create -role harvest-rest-role -access readonly -api /api/storage/namespaces
security login rest-role create -role harvest-rest-role -access readonly -api /api/storage/qtrees
security login rest-role create -role harvest-rest-role -access readonly -api /api/storage/qos/policies
security login rest-role create -role harvest-rest-role -access readonly -api /api/storage/qos/workloads
security login rest-role create -role harvest-rest-role -access readonly -api /api/storage/quota/reports
security login rest-role create -role harvest-rest-role -access readonly -api /api/storage/shelves
security login rest-role create -role harvest-rest-role -access readonly -api /api/storage/volumes
security login rest-role create -role harvest-rest-role -access readonly -api /api/support/auto-update
security login rest-role create -role harvest-rest-role -access readonly -api /api/support/autosupport
security login rest-role create -role harvest-rest-role -access readonly -api /api/support/ems/destinations
security login rest-role create -role harvest-rest-role -access readonly -api /api/support/ems/events
security login rest-role create -role harvest-rest-role -access readonly -api /api/svm/peers
security login rest-role create -role harvest-rest-role -access readonly -api /api/svm/svms
Thanks @mamoep!
There are 20 permissions errors (see below) when running Harvest with Rest and RestPerf collectors when using the permission list from above.
Three of the "permission denied" failures are from public APIs and the other 17 are from private CLIs. The three public APIs that failed are:
security login rest-role create -role h-tmp-role -access readonly -api /api/protocols/s3/buckets
Error: command failed: Specified URI path is invalid or not supported. Verify that the URI contains only valid characters.
- /api/storage/qtrees
This one works fine
- /api/support/autosupport
This one is in your list, was added on this cluster, but still returns a 403. Bug?
```bash
rg 'Permission denied' /tmp/n.t | rg -o 'API:.* Poller' | cut -d' ' -f2 | cut -d'?' -f1 | sort | uniq
/api/private/cli/aggr
/api/private/cli/cluster/date
/api/private/cli/disk
/api/private/cli/network/interface
/api/private/cli/node
/api/private/cli/qos/adaptive-policy-group
/api/private/cli/qos/policy-group
/api/private/cli/qos/workload
/api/private/cli/snapmirror
/api/private/cli/snapshot/policy
/api/private/cli/storage/failover
/api/private/cli/storage/shelf
/api/private/cli/system/chassis/fru
/api/private/cli/system/health/subsystem
/api/private/cli/volume
/api/private/cli/vserver
/api/private/cli/vserver/object-store-server/bucket/policy
/api/protocols/s3/buckets
/api/storage/qtrees
/api/support/autosupport
Creating a new role (h-tmp-role
) with your list + the one missing qtree role results in security login rest-role show -role h-tmp-role
displaying 42 entries and security login role show -role h-tmp-role
displays 167 entries.
I think I forgot the most basic for private cli. :)
security login rest-role create -role harvest-rest-role -access readonly -api /api/private/cli
s3 is a known issue: https://mysupport.netapp.com/site/bugs-online/product/ONTAP/JiraNgage/CONTAP-210232
I added the missing parts to my initial list
Do you still consider it "least privilege" if you're required to add /api/private/cli
?
Adding /api/private/cli
is identical to running security login role create -role harvest2-role -access readonly -cmddirname "DEFAULT"
, which is adding readonly access for all CLI apis.
Left side is before adding /api/private/cli
and right side is after
Guess your right, that defeats the purpose. :(
Looks different in my simulator. Which ONTAP version are you running?
sim01::*> security login role show -vserver sim01 -role harvest-rest-role
Role Command/ Access
Vserver Name Directory Query Level
---------- ------------- --------- ----------------------------------- --------
sim01 harvest-rest-role
DEFAULT none
system node run readonly
system node systemshell readonly
sim01::*> security login rest-role show -vserver sim01 -role harvest-rest-role
Role Access
Vserver Name API Level
---------- ------------- ------------------- ------
sim01 harvest-rest-role
/api/private/cli readonly
version
NetApp Release Mightysquirrel__9.15.1: Fri May 24 05:03:10 UTC 2024
So mapping inconsistencies between different ONTAP versions. This whole REST thing is a mess... Do at least the "Permission denied" errors disappear when adding the private cli endpoint?
Perhaps what you are seeing is a simulator issue. I'll try on another cluster and report back.
If I add /api/private/cli
(which adds the wildcard RBAC), more things work. Here are the other misses:
/api/name-services/ldap
/api/name-services/nis
/api/storage/quota/reports
/api/svm/peers
This API, used by the Rest:Health plugin, still fails which is confusing
/api/private/support/alerts
Adding /api/private/cli
on a NetApp Release 9.13.1P3: Wed Sep 20 01:37:11 UTC 2023
cluster does NOT touch the DEFAULT command.
Adding /api/private/cli
on a NetApp Release 9.14.1P2: Tue Mar 19 23:28:08 UTC 2024
cluster changes the DEFAULT command from none to readonly.
/api/private/support/alerts
is not equal /api/private/cli
. But it can be added to the rest-role. I guess they hide some stuff in there. :)
Can you talk to some ONTAP people about the DEFAULT command getting added in 9.15?
hi @mamoep I heard back from ONTAP. This change was intentional and not a bug (burt 1496317). When you specify the endpoint /api/private/cli
as readonly, the DEFAULT access for the traditional role will also change to readonly.
I wonder if the behavior you want could be achieved by not adding /api/private/cli
and instead adding each child tree individually /api/private/cli/aggr
, /api/private/cli/cluster/date
, /api/private/cli/disk
, etc?
I have confirmed that if you add a subpath of api/private/cli
the DEFAULT command access remains none.
security login role show h-tmp-role
Role Command/ Access
Vserver Name Directory Query Level
---------- ------------- --------- ----------------------------------- --------
umeng-aff300-01-02
h-tmp-role DEFAULT none
snapmirror object-store config readonly
storage aggregate object-store config readonly
3 entries were displayed.
security login rest-role create -role h-tmp-role -access readonly -api /api/private/cli/system/chassis/fru
umeng-aff300-01-02::> security login role show h-tmp-role
Role Command/ Access
Vserver Name Directory Query Level
---------- ------------- --------- ----------------------------------- --------
umeng-aff300-01-02
h-tmp-role DEFAULT none
snapmirror object-store config readonly
storage aggregate object-store config readonly
system chassis fru readonly
4 entries were displayed.
Subpath doesn't work on 9.13.1
security login rest-role create -role harvest-rest-role -api /api/private/cli/system/chassis/fru -access readonly
Error: command failed: URI does not exist.
The BURT you mentioned is not public readable. I speculate global readonly and subpath availability go hand in hand. Could you confirm this with the ONTAP team? Will this change be back-ported to 9.13 or 9.14?
@mamoep I'm waiting to hear back from ONTAP on whether this change can be back-ported and will update this issue when I have an answer.
Is your feature request related to a problem? Please describe. The documentation has a detailed list of permissions needed for legacy roles. For rest-role it is just a readonly for the whole api.
Describe the solution you'd like Please document a least-privilege approach for
security login rest-role create
command.