Open ab9876543210 opened 1 year ago
As further info, here the logs from server-container-debug.log:
22-11-21 14:44:39 | cluster_service_2_x:897 - _create_cluster_async | Request Id: 74202936-ba0c-4022-8891-0b421aee2a40 | DEBUG :: Setting metadata on cluster vApp 'obs-test-2211P11345-2'
22-11-21 14:44:45 | cluster_service_2_x:917 - _create_cluster_async | Request Id: 74202936-ba0c-4022-8891-0b421aee2a40 | DEBUG :: Creating control plane node for cluster 'obs-test-2211P11345-2' (urn:vcloud:entity:cse:nativeCluster:cf236302-a5fa-46f2-b699-7cd77b66ddb3)
22-11-21 14:44:45 | cluster_service_2_x:2172 - _update_task | Request Id: 74202936-ba0c-4022-8891-0b421aee2a40 | DEBUG :: Sending behavior response:{'type': 'BEHAVIOR_RESPONSE', 'headers': {'taskId': '06321893-3cf3-48e2-84eb-bd559de0517b', 'entityId': 'urn:vcloud:entity:cse:nativeCluster:cf236302-a5fa-46f2-b699-7cd77b66ddb3', 'contentType': 'application/vnd.vmware.vcloud.task+json'}, 'payload': '{"status": "running", "operation": "Creating control plane node for cluster \'obs-test-2211P11345-2\' (urn:vcloud:entity:cse:nativeCluster:cf236302-a5fa-46f2-b699-7cd77b66ddb3)"}'}
22-11-21 14:44:45 | mqtt_publisher:116 - send_response | Request Id: 74202936-ba0c-4022-8891-0b421aee2a40 | DEBUG :: publish return (rc, msg_id): (0, 37210)
22-11-21 14:44:48 | cluster_service_2_x:2593 - _add_nodes | Request Id: 74202936-ba0c-4022-8891-0b421aee2a40 | DEBUG :: Found sizing policy with name System Default on the VDC FCH-ITR_ISN7
22-11-21 14:44:48 | cluster_service_2_x:2675 - _add_nodes | Request Id: 74202936-ba0c-4022-8891-0b421aee2a40 | ERROR :: Status code: 403/ACCESS_TO_RESOURCE_IS_FORBIDDEN, [ 0b2e9e0d-f83b-4b8a-b45e-bcd500e818c7 ] Either you need some or all of the following rights [Base] to perform operations [VAPP_VM_VIEW] for 8afc3401-9a9a-4e57-933d-be534cec5f1f or the target entity is invalid. (request id: 0b2e9e0d-f83b-4b8a-b45e-bcd500e818c7)
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/container_service_extension/rde/backend/cluster_service_2_x.py", line 2632, in _add_nodes
task = vapp.add_vms(specs, power_on=False)
File "/usr/local/lib/python3.7/site-packages/pyvcloud/vcd/vapp.py", line 1032, in add_vms
EntityType.RECOMPOSE_VAPP_PARAMS.value, params)
File "/usr/local/lib/python3.7/site-packages/pyvcloud/vcd/client.py", line 1537, in post_linked_resource
media_type, extra_headers=extra_headers)
File "/usr/local/lib/python3.7/site-packages/pyvcloud/vcd/client.py", line 1520, in post_resource
extra_headers=extra_headers)
File "/usr/local/lib/python3.7/site-packages/pyvcloud/vcd/client.py", line 1267, in _do_request
_objectify_response(response, objectify_results))
File "/usr/local/lib/python3.7/site-packages/pyvcloud/vcd/client.py", line 1278, in _response_code_to_exception
raise AccessForbiddenException(sc, request_id, objectify_response)
pyvcloud.vcd.exceptions.AccessForbiddenException: Status code: 403/ACCESS_TO_RESOURCE_IS_FORBIDDEN, [ 0b2e9e0d-f83b-4b8a-b45e-bcd500e818c7 ] Either you need some or all of the following rights [Base] to perform operations [VAPP_VM_VIEW] for 8afc3401-9a9a-4e57-933d-be534cec5f1f or the target entity is invalid. (request id: 0b2e9e0d-f83b-4b8a-b45e-bcd500e818c7)
22-11-21 14:44:48 | cluster_service_2_x:940 - _create_cluster_async | Request Id: 74202936-ba0c-4022-8891-0b421aee2a40 | ERROR :: failure on creating nodes ['mstr-vmg7']
Describe the bug
We are on VCD 10.3.3.20027910 and CSE 3.1.4, and are having issues deploying native clusters with limited-privilege (customized vApp Author + CSE rights; we are calling this "Orchestrator") users. The same operation succeeds with higher-privilege role (customized Organization Administrator + CSE rights). Before upgrading to CSE 3.1.4, the Orchestrator role could create CSE clusters just fine. All CSE rights were/are assigned to it, along with necessary vApp rights.
Currently, deployments are failing with this error:
This looks like an error on CSE end only, as the user can create vApps just fine in the same Org-vDC where we are attempting to create CSE clusters.
It seems the same error of https://github.com/vmware/container-service-extension/pull/1360 which was for TKGm clusters, and supposedly fixed on 3.1.4 ..
Reproduction steps
1. 2. 3. ...
Expected behavior
Native cluster deployments should succeed using an account which has the necessary vApp/CSE clusters creation rights.
Additional context
Limited privilege role rights: