search

vmware / container-service-extension

Container Service for VMware vCloud Director
https://vmware.github.io/container-service-extension
Other
78 stars 52 forks source link

Cluster creation request failed. Please contact your provider if this problem persists. Http failure response for https://vFQDN/api/cse/3.0/clusters: 500 Server Error #1358

Open sas-de opened 2 years ago

sas-de commented 2 years ago

Describe the bug

Problem by Create a new VMware Tanzu Kubernetes Grid Cluster in VCD over CSE

Error Message in VCD: Cluster creation request failed. Please contact your provider if this problem persists. Http failure response for https://vcd.dc.schwunk-it.de/api/cse/3.0/clusters: 500 Server Error

Log Files: 22-05-17 10:24:12 | request_dispatcher:894 - process_request | Request Id: 565fe23c-f87e-4a85-adce-53ef64e20758 | DEBUG :: Incoming request message: {"id": "2842f97c-0181-42f7-a91b-019e2d0cd8f4", "method": "GET", "requestUri": "/api/cse/templates/tkgm", "queryString": null, "protocol": "HTTP/1.1", "scheme": "https", "remoteAddr": "192.168.254.1", "remotePort": 52554, "localAddr": "192.168.254.31", "localPort": 443, "headers": {"Cookie": "_ga=GA1.2.130519610.1639845461; vcloud_jwt=eyJhbGciOiJSUzI1NiJ9.eyJzdWIiOiJhZG1pbmlzdHJhdG9yIiwiaXNzIjoiYTkzYzlkYjktNzQ3MS0zMTkyLThkMDktYThmN2VlZGE4NWY5QDAzM2UzMjJmLWZkZGEtNDJkOS1iOWJmLTM5MjI3MzNkOTMxNyIsImV4cCI6MTY1Mjg2OTI3MiwidmVyc2lvbiI6InZjbG91ZF8xLjAiLCJqdGkiOiI5ZTYzN2NiNDg5Yjk0YzFjOWEzYmQzZTdmYTQwMmM0YiJ9.FvUJhmWnJ2p7u8nlq7lfjDQJKJakPAuUxazIXcBww2Qfprp7ExzRz-VwG7j4Y1D09_IcmpZX8iJGkCxT7UVyHsQesWLHuraDMuNTMHpKJfA2sQJ0733ZC4q5C-XXUJGBXu0mxEex3mxchAjsm1jw0jxjAWlSvpm4pfNGkGtnBwXQuwyJHC6kbMxIRSsKiKjAhJ91iyB6TmpPRfu6WPlKYq1lsimbhBh-WVyV3TFkalEpQiALshnFJ_t6qMdQpROXg67MVGG1mfPRhDisU3oI4k6WYK_phFzz_ENAGh25ODMKpeasskOvh91R16dFnciVlNk87_ImM0Nn5lPYtK6--w; vcloud_session_id=9e637cb489b94c1c9a3bd3e7fa402c4b; _pk_ses.1.928c=0; _pk_id.1.928c=b42eee8f5b04b65f.1650899400.12.1652783035.1652782931..e167dd23fb28b446bfcec22138c190a5805e5757b2109895657d6b5975b97d4e", "Accept": "application/+json;version=36.0, application/json;version=36.0", "Connection": "keep-alive", "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.54 Safari/537.36", "Referer": "https://vcd.dc.schwunk-it.de/provider/plugins/Vk13YXJl/cse", "Sec-Fetch-Site": "same-origin", "Sec-Fetch-Dest": "empty", "Host": "vcd.dc.schwunk-it.de", "Accept-Encoding": "gzip, deflate, br", "Sec-Fetch-Mode": "cors", "Authorization": [[REDACTED], "sec-ch-ua": "\" Not A;Brand\";v=\"99\", \"Chromium\";v=\"101\", \"Google Chrome\";v=\"101\"", "sec-ch-ua-mobile": "?0", "sec-ch-ua-platform": "\"macOS\"", "Accept-Language": "en-US,en;q=0.9,de;q=0.8,de-DE;q=0.7", "Content-Type": "application/+json"}, "body": "", "statusCode": 0, "request": true} 22-05-17 10:24:12 | request_dispatcher:1045 - process_request | Request Id: 565fe23c-f87e-4a85-adce-53ef64e20758 | DEBUG :: body content: [{'catalog': 'cse', 'catalog_item': 'ubuntu-2004-kube-v1.21.2+vmware.1-tkg.1-7832907791984498322', 'cni': 'antrea', 'cni_version': '0.0.0', 'cse_version': '3.1.3', 'deprecated': None, 'description': None, 'docker_version': None, 'container_runtime': 'containerd', 'container_runtime_version': 'v1.4.6+vmware.1', 'is_default': False, 'kind': 'TKGm', 'kubernetes': 'TKGm', 'kubernetes_version': 'v1.21.2+vmware.1', 'name': 'ubuntu-2004-kube-v1.21.2+vmware.1-tkg.1-7832907791984498322', 'os': 'ubuntu-20.04', 'revision': 1}, {'catalog': 'cse', 'catalog_item': 'ubuntu-2004-kube-v1.19.12+vmware.1-tkg.1-15841320193950299489', 'cni': 'antrea', 'cni_version': '0.0.0', 'cse_version': '3.1.3', 'deprecated': None, 'description': None, 'docker_version': None, 'container_runtime': 'containerd', 'container_runtime_version': 'v1.4.6+vmware.1', 'is_default': False, 'kind': 'TKGm', 'kubernetes': 'TKGm', 'kubernetes_version': 'v1.19.12+vmware.1', 'name': 'ubuntu-2004-kube-v1.19.12+vmware.1-tkg.1-15841320193950299489', 'os': 'ubuntu-20.04', 'revision': 1}] 22-05-17 10:24:12 | request_dispatcher:1056 - process_request | Request Id: 565fe23c-f87e-4a85-adce-53ef64e20758 | DEBUG :: Outgoing response: {'status_code': 200, 'body': [{'catalog': 'cse', 'catalog_item': 'ubuntu-2004-kube-v1.21.2+vmware.1-tkg.1-7832907791984498322', 'cni': 'antrea', 'cni_version': '0.0.0', 'cse_version': '3.1.3', 'deprecated': None, 'description': None, 'docker_version': None, 'container_runtime': 'containerd', 'container_runtime_version': 'v1.4.6+vmware.1', 'is_default': False, 'kind': 'TKGm', 'kubernetes': 'TKGm', 'kubernetes_version': 'v1.21.2+vmware.1', 'name': 'ubuntu-2004-kube-v1.21.2+vmware.1-tkg.1-7832907791984498322', 'os': 'ubuntu-20.04', 'revision': 1}, {'catalog': 'cse', 'catalog_item': 'ubuntu-2004-kube-v1.19.12+vmware.1-tkg.1-15841320193950299489', 'cni': 'antrea', 'cni_version': '0.0.0', 'cse_version': '3.1.3', 'deprecated': None, 'description': None, 'docker_version': None, 'container_runtime': 'containerd', 'container_runtime_version': 'v1.4.6+vmware.1', 'is_default': False, 'kind': 'TKGm', 'kubernetes': 'TKGm', 'kubernetes_version': 'v1.19.12+vmware.1', 'name': 'ubuntu-2004-kube-v1.19.12+vmware.1-tkg.1-15841320193950299489', 'os': 'ubuntu-20.04', 'revision': 1}]} 22-05-17 10:24:12 | mqtt_consumer:73 - process_mqtt_message | Request Id: 565fe23c-f87e-4a85-adce-53ef64e20758 | DEBUG :: Received message with request_id: 565fe23c-f87e-4a85-adce-53ef64e20758, mid: 4, and msg json: {'id': '2842f97c-0181-42f7-a91b-019e2d0cd8f4', 'method': 'GET', 'requestUri': '/api/cse/templates/tkgm', 'queryString': None, 'protocol': 'HTTP/1.1', 'scheme': 'https', 'remoteAddr': '192.168.254.1', 'remotePort': 52554, 'localAddr': '192.168.254.31', 'localPort': 443, 'headers': {'Cookie': '_ga=GA1.2.130519610.1639845461; vcloud_jwt=eyJhbGciOiJSUzI1NiJ9.eyJzdWIiOiJhZG1pbmlzdHJhdG9yIiwiaXNzIjoiYTkzYzlkYjktNzQ3MS0zMTkyLThkMDktYThmN2VlZGE4NWY5QDAzM2UzMjJmLWZkZGEtNDJkOS1iOWJmLTM5MjI3MzNkOTMxNyIsImV4cCI6MTY1Mjg2OTI3MiwidmVyc2lvbiI6InZjbG91ZF8xLjAiLCJqdGkiOiI5ZTYzN2NiNDg5Yjk0YzFjOWEzYmQzZTdmYTQwMmM0YiJ9.FvUJhmWnJ2p7u8nlq7lfjDQJKJakPAuUxazIXcBww2Qfprp7ExzRz-VwG7j4Y1D09_IcmpZX8iJGkCxT7UVyHsQesWLHuraDMuNTMHpKJfA2sQJ0733ZC4q5C-XXUJGBXu0mxEex3mxchAjsm1jw0jxjAWlSvpm4pfNGkGtnBwXQuwyJHC6kbMxIRSsKiKjAhJ91iyB6TmpPRfu6WPlKYq1lsimbhBh-WVyV3TFkalEpQiALshnFJ_t6qMdQpROXg67MVGG1mfPRhDisU3oI4k6WYK_phFzz_ENAGh25ODMKpeasskOvh91R16dFnciVlNk87_ImM0Nn5lPYtK6--w; vcloud_session_id=9e637cb489b94c1c9a3bd3e7fa402c4b; _pk_ses.1.928c=0; _pk_id.1.928c=b42eee8f5b04b65f.1650899400.12.1652783035.1652782931..e167dd23fb28b446bfcec22138c190a5805e5757b2109895657d6b5975b97d4e', 'Accept': 'application/+json;version=36.0, application/json;version=36.0', 'Connection': 'keep-alive', 'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.54 Safari/537.36', 'Referer': 'https://vcd.dc.schwunk-it.de/provider/plugins/Vk13YXJl/cse', 'Sec-Fetch-Site': 'same-origin', 'Sec-Fetch-Dest': 'empty', 'Host': 'vcd.dc.schwunk-it.de', 'Accept-Encoding': 'gzip, deflate, br', 'Sec-Fetch-Mode': 'cors', 'Authorization': '[REDACTED]', 'sec-ch-ua': '" Not A;Brand";v="99", "Chromium";v="101", "Google Chrome";v="101"', 'sec-ch-ua-mobile': '?0', 'sec-ch-ua-platform': '"macOS"', 'Accept-Language': 'en-US,en;q=0.9,de;q=0.8,de-DE;q=0.7', 'Content-Type': 'application/+json'}, 'body': '', 'statusCode': 0, 'request': True} 22-05-17 10:24:12 | mqtt_publisher:116 - send_response | Request Id: 565fe23c-f87e-4a85-adce-53ef64e20758 | DEBUG :: publish return (rc, msg_id): (0, 5) 22-05-17 10:24:12 | mqtt_consumer:85 - process_mqtt_message | Request Id: 565fe23c-f87e-4a85-adce-53ef64e20758 | DEBUG :: MQTT response: {'type': 'API_RESPONSE', 'headers': {'requestId': '565fe23c-f87e-4a85-adce-53ef64e20758'}, 'httpResponse': {'statusCode': 200, 'headers': {'Content-Type': 'application/json', 'Content-Length': 1047}, 'body': 'W3siY2F0YWxvZyI6ICJjc2UiLCAiY2F0YWxvZ19pdGVtIjogInVidW50dS0yMDA0LWt1YmUtdjEuMjEuMit2bXdhcmUuMS10a2cuMS03ODMyOTA3NzkxOTg0NDk4MzIyIiwgImNuaSI6ICJhbnRyZWEiLCAiY25pX3ZlcnNpb24iOiAiMC4wLjAiLCAiY3NlX3ZlcnNpb24iOiAiMy4xLjMiLCAiZGVwcmVjYXRlZCI6IG51bGwsICJkZXNjcmlwdGlvbiI6IG51bGwsICJkb2NrZXJfdmVyc2lvbiI6IG51bGwsICJjb250YWluZXJfcnVudGltZSI6ICJjb250YWluZXJkIiwgImNvbnRhaW5lcl9ydW50aW1lX3ZlcnNpb24iOiAidjEuNC42K3Ztd2FyZS4xIiwgImlzX2RlZmF1bHQiOiBmYWxzZSwgImtpbmQiOiAiVEtHbSIsICJrdWJlcm5ldGVzIjogIlRLR20iLCAia3ViZXJuZXRlc192ZXJzaW9uIjogInYxLjIxLjIrdm13YXJlLjEiLCAibmFtZSI6ICJ1YnVudHUtMjAwNC1rdWJlLXYxLjIxLjIrdm13YXJlLjEtdGtnLjEtNzgzMjkwNzc5MTk4NDQ5ODMyMiIsICJvcyI6ICJ1YnVudHUtMjAuMDQiLCAicmV2aXNpb24iOiAxfSwgeyJjYXRhbG9nIjogImNzZSIsICJjYXRhbG9nX2l0ZW0iOiAidWJ1bnR1LTIwMDQta3ViZS12MS4xOS4xMit2bXdhcmUuMS10a2cuMS0xNTg0MTMyMDE5Mzk1MDI5OTQ4OSIsICJjbmkiOiAiYW50cmVhIiwgImNuaV92ZXJzaW9uIjogIjAuMC4wIiwgImNzZV92ZXJzaW9uIjogIjMuMS4zIiwgImRlcHJlY2F0ZWQiOiBudWxsLCAiZGVzY3JpcHRpb24iOiBudWxsLCAiZG9ja2VyX3ZlcnNpb24iOiBudWxsLCAiY29udGFpbmVyX3J1bnRpbWUiOiAiY29udGFpbmVyZCIsICJjb250YWluZXJfcnVudGltZV92ZXJzaW9uIjogInYxLjQuNit2bXdhcmUuMSIsICJpc19kZWZhdWx0IjogZmFsc2UsICJraW5kIjogIlRLR20iLCAia3ViZXJuZXRlcyI6ICJUS0dtIiwgImt1YmVybmV0ZXNfdmVyc2lvbiI6ICJ2MS4xOS4xMit2bXdhcmUuMSIsICJuYW1lIjogInVidW50dS0yMDA0LWt1YmUtdjEuMTkuMTIrdm13YXJlLjEtdGtnLjEtMTU4NDEzMjAxOTM5NTAyOTk0ODkiLCAib3MiOiAidWJ1bnR1LTIwLjA0IiwgInJldmlzaW9uIjogMX1d'}} 22-05-17 10:24:59 | request_dispatcher:894 - process_request | Request Id: fe8f8703-833e-4033-b50c-94d0d15d297d | DEBUG :: Incoming request message: {"id": "e09bcf0d-50a8-441c-aec2-31484de814a8", "method": "POST", "requestUri": "/api/cse/3.0/clusters", "queryString": null, "protocol": "HTTP/1.1", "scheme": "https", "remoteAddr": "192.168.254.1", "remotePort": 52479, "localAddr": "192.168.254.31", "localPort": 443, "headers": {"Origin": "https://vcd.dc.schwunk-it.de", "Cookie": "_ga=GA1.2.130519610.1639845461; vcloud_jwt=eyJhbGciOiJSUzI1NiJ9.eyJzdWIiOiJhZG1pbmlzdHJhdG9yIiwiaXNzIjoiYTkzYzlkYjktNzQ3MS0zMTkyLThkMDktYThmN2VlZGE4NWY5QDAzM2UzMjJmLWZkZGEtNDJkOS1iOWJmLTM5MjI3MzNkOTMxNyIsImV4cCI6MTY1Mjg2OTI3MiwidmVyc2lvbiI6InZjbG91ZF8xLjAiLCJqdGkiOiI5ZTYzN2NiNDg5Yjk0YzFjOWEzYmQzZTdmYTQwMmM0YiJ9.FvUJhmWnJ2p7u8nlq7lfjDQJKJakPAuUxazIXcBww2Qfprp7ExzRz-VwG7j4Y1D09_IcmpZX8iJGkCxT7UVyHsQesWLHuraDMuNTMHpKJfA2sQJ0733ZC4q5C-XXUJGBXu0mxEex3mxchAjsm1jw0jxjAWlSvpm4pfNGkGtnBwXQuwyJHC6kbMxIRSsKiKjAhJ91iyB6TmpPRfu6WPlKYq1lsimbhBh-WVyV3TFkalEpQiALshnFJ_t6qMdQpROXg67MVGG1mfPRhDisU3oI4k6WYK_phFzz_ENAGh25ODMKpeasskOvh91R16dFnciVlNk87_ImM0Nn5lPYtK6--w; vcloud_session_id=9e637cb489b94c1c9a3bd3e7fa402c4b; _pk_ses.1.928c=0; _pk_id.1.928c=b42eee8f5b04b65f.1650899400.12.1652783099.1652782931..e167dd23fb28b446bfcec22138c190a5805e5757b2109895657d6b5975b97d4e", "Accept": "application/+json;version=36.0, application/json;version=36.0", "Connection": "keep-alive", "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.54 Safari/537.36", "Referer": "https://vcd.dc.schwunk-it.de/provider/plugins/Vk13YXJl/cse", "Sec-Fetch-Site": "same-origin", "Sec-Fetch-Dest": "empty", "Host": "vcd.dc.schwunk-it.de", "Accept-Encoding": "gzip, deflate, br", "Sec-Fetch-Mode": "cors", "Authorization": [[REDACTED], "sec-ch-ua": "\" Not A;Brand\";v=\"99\", \"Chromium\";v=\"101\", \"Google Chrome\";v=\"101\"", "sec-ch-ua-mobile": "?0", "X-VMWARE-VCLOUD-TENANT-CONTEXT": "6d101c73-a61a-4173-b1e4-8e5569a9055b", "sec-ch-ua-platform": "\"macOS\"", "Accept-Language": "en-US,en;q=0.9,de;q=0.8,de-DE;q=0.7", "Content-Length": "925", "Content-Type": "application/+json"}, "body": "eyJhcGlWZXJzaW9uIjoiY3NlLnZtd2FyZS5jb20vdjIuMSIsImtpbmQiOiJUS0dtIiwibWV0YWRhdGEiOnsiYWRkaXRpb25hbFByb3BlcnRpZXMiOnRydWUsIm9yZ05hbWUiOiJEZW1vIiwidmlydHVhbERhdGFDZW50ZXJOYW1lIjoiRGVtby1WREMiLCJuYW1lIjoiazhzIiwic2l0ZSI6Imh0dHBzOi8vdmNkLmRjLnNjaHd1bmstaXQuZGUifSwic3BlYyI6eyJhZGRpdGlvbmFsUHJvcGVydGllcyI6dHJ1ZSwidG9wb2xvZ3kiOnsiY29udHJvbFBsYW5lIjp7ImNvdW50IjoxLCJzaXppbmdDbGFzcyI6Ims4cyIsInN0b3JhZ2VQcm9maWxlIjoidGFuenUiLCJjcHUiOm51bGwsIm1lbW9yeSI6bnVsbH0sIndvcmtlcnMiOnsiY291bnQiOjIsInNpemluZ0NsYXNzIjoiazhzIiwic3RvcmFnZVByb2ZpbGUiOiJ0YW56dSIsImNwdSI6bnVsbCwibWVtb3J5IjpudWxsfSwibmZzIjp7ImNvdW50IjowLCJzaXppbmdDbGFzcyI6bnVsbCwic3RvcmFnZVByb2ZpbGUiOm51bGx9fSwic2V0dGluZ3MiOnsib3ZkY05ldHdvcmsiOiJEZW1vIiwicm9sbGJhY2tPbkZhaWx1cmUiOnRydWUsIm5ldHdvcmsiOnsiZXhwb3NlIjp0cnVlLCJwb2RzIjp7ImNpZHJCbG9ja3MiOlsiMTAwLjk2LjAuMC8xMSJdfSwic2VydmljZXMiOnsiY2lkckJsb2NrcyI6WyIxMDAuNjQuMC4wLzEzIl19fSwiY3NpIjpbeyJkZWZhdWx0SzhzU3RvcmFnZUNsYXNzIjp7InZjZFN0b3JhZ2VQcm9maWxlTmFtZSI6InRhbnp1IiwiazhzU3RvcmFnZUNsYXNzTmFtZSI6InRhbnp1IiwidXNlRGVsZXRlUmVjbGFpbVBvbGljeSI6dHJ1ZSwiZmlsZXN5c3RlbSI6ImV4dDQifX1dfSwiZGlzdHJpYnV0aW9uIjp7InRlbXBsYXRlTmFtZSI6InVidW50dS0yMDA0LWt1YmUtdjEuMjEuMit2bXdhcmUuMS10a2cuMS03ODMyOTA3NzkxOTg0NDk4MzIyIiwidGVtcGxhdGVSZXZpc2lvbiI6MX19fQ==", "statusCode": 0, "request": true} 22-05-17 10:24:59 | request_dispatcher:1020 - process_request | Request Id: fe8f8703-833e-4033-b50c-94d0d15d297d | DEBUG :: request body: {'apiVersion': 'cse.vmware.com/v2.1', 'kind': 'TKGm', 'metadata': {'additionalProperties': True, 'orgName': 'Demo', 'virtualDataCenterName': 'Demo-VDC', 'name': 'k8s', 'site': 'https://vcd.dc.schwunk-it.de'}, 'spec': {'additionalProperties': True, 'topology': {'controlPlane': {'count': 1, 'sizingClass': 'k8s', 'storageProfile': 'tanzu', 'cpu': None, 'memory': None}, 'workers': {'count': 2, 'sizingClass': 'k8s', 'storageProfile': 'tanzu', 'cpu': None, 'memory': None}, 'nfs': {'count': 0, 'sizingClass': None, 'storageProfile': None}}, 'settings': {'ovdcNetwork': 'Demo', 'rollbackOnFailure': True, 'network': {'expose': True, 'pods': {'cidrBlocks': ['100.96.0.0/11']}, 'services': {'cidrBlocks': ['100.64.0.0/13']}}, 'csi': [{'defaultK8sStorageClass': {'vcdStorageProfileName': 'tanzu', 'k8sStorageClassName': 'tanzu', 'useDeleteReclaimPolicy': True, 'filesystem': 'ext4'}}]}, 'distribution': {'templateName': 'ubuntu-2004-kube-v1.21.2+vmware.1-tkg.1-7832907791984498322', 'templateRevision': 1}}} 22-05-17 10:25:00 | entity_service:56 - exception_handler_wrapper | Request Id: fe8f8703-833e-4033-b50c-94d0d15d297d | ERROR :: [ adfa858a-b368-410e-b4b6-080fd322a493 ] This operation is denied. Traceback (most recent call last): File "/opt/cse/python/lib/python3.7/site-packages/container_service_extension/rde/common/entity_service.py", line 52, in exception_handler_wrapper result = func(*args, *kwargs) File "/opt/cse/python/lib/python3.7/site-packages/container_service_extension/rde/common/entity_service.py", line 113, in create_entity return_response_headers=is_request_async) File "/opt/cse/python/lib/python3.7/site-packages/container_service_extension/lib/cloudapi/cloudapi_client.py", line 140, in do_request response.raise_for_status() File "/opt/cse/python/lib/python3.7/site-packages/requests/models.py", line 960, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://vcd.dc.schwunk-it.de/cloudapi/1.0.0/entityTypes/urn:vcloud:type:cse:nativeCluster:2.1.0 22-05-17 10:25:00 | request_utils:166 - exception_handler_wrapper | Request Id: fe8f8703-833e-4033-b50c-94d0d15d297d | ERROR :: [ adfa858a-b368-410e-b4b6-080fd322a493 ] This operation is denied. 22-05-17 10:25:00 | exception_handler:53 - exception_handler_wrapper | Request Id: fe8f8703-833e-4033-b50c-94d0d15d297d | ERROR :: Traceback (most recent call last): File "/opt/cse/python/lib/python3.7/site-packages/container_service_extension/rde/common/entity_service.py", line 52, in exception_handler_wrapper result = func(args, **kwargs) File "/opt/cse/python/lib/python3.7/site-packages/container_service_extension/rde/common/entity_service.py", line 113, in create_entity return_response_headers=is_request_async) File "/opt/cse/python/lib/python3.7/site-packages/container_service_extension/lib/cloudapi/cloudapi_client.py", line 140, in do_request response.raise_for_status() File "/opt/cse/python/lib/python3.7/site-packages/requests/models.py", line 960, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://vcd.dc.schwunk-it.de/cloudapi/1.0.0/entityTypes/urn:vcloud:type:cse:nativeCluster:2.1.0

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/opt/cse/python/lib/python3.7/site-packages/container_service_extension/exception/exception_handler.py", line 37, in exception_handler_wrapper result = func(*args, kwargs) File "/opt/cse/python/lib/python3.7/site-packages/container_service_extension/server/request_dispatcher.py", line 1044, in process_request body_content = handler_method(request_data, operation_ctx) File "/opt/cse/python/lib/python3.7/site-packages/container_service_extension/lib/telemetry/telemetry_handler.py", line 113, in wrapper raise err File "/opt/cse/python/lib/python3.7/site-packages/container_service_extension/lib/telemetry/telemetry_handler.py", line 107, in wrapper ret_value = func(*args, *kwargs) File "/opt/cse/python/lib/python3.7/site-packages/container_service_extension/server/request_handlers/request_utils.py", line 167, in exception_handler_wrapper raise error File "/opt/cse/python/lib/python3.7/site-packages/container_service_extension/server/request_handlers/request_utils.py", line 161, in exception_handler_wrapper result = func(args, kwargs) File "/opt/cse/python/lib/python3.7/site-packages/container_service_extension/server/request_handlers/cluster_handler.py", line 87, in cluster_create is_request_async=True) File "/opt/cse/python/lib/python3.7/site-packages/container_service_extension/rde/common/entity_service.py", line 59, in exception_handler_wrapper minor_error_code=MinorErrorCode.DEFAULT_ERROR_CODE) container_service_extension.exception.exceptions.DefEntityServiceError: [ adfa858a-b368-410e-b4b6-080fd322a493 ] This operation is denied.

22-05-17 10:25:00 | mqtt_consumer:73 - process_mqtt_message | Request Id: fe8f8703-833e-4033-b50c-94d0d15d297d | DEBUG :: Received message with request_id: fe8f8703-833e-4033-b50c-94d0d15d297d, mid: 5, and msg json: {'id': 'e09bcf0d-50a8-441c-aec2-31484de814a8', 'method': 'POST', 'requestUri': '/api/cse/3.0/clusters', 'queryString': None, 'protocol': 'HTTP/1.1', 'scheme': 'https', 'remoteAddr': '192.168.254.1', 'remotePort': 52479, 'localAddr': '192.168.254.31', 'localPort': 443, 'headers': {'Origin': 'https://vcd.dc.schwunk-it.de', 'Cookie': '_ga=GA1.2.130519610.1639845461; vcloud_jwt=eyJhbGciOiJSUzI1NiJ9.eyJzdWIiOiJhZG1pbmlzdHJhdG9yIiwiaXNzIjoiYTkzYzlkYjktNzQ3MS0zMTkyLThkMDktYThmN2VlZGE4NWY5QDAzM2UzMjJmLWZkZGEtNDJkOS1iOWJmLTM5MjI3MzNkOTMxNyIsImV4cCI6MTY1Mjg2OTI3MiwidmVyc2lvbiI6InZjbG91ZF8xLjAiLCJqdGkiOiI5ZTYzN2NiNDg5Yjk0YzFjOWEzYmQzZTdmYTQwMmM0YiJ9.FvUJhmWnJ2p7u8nlq7lfjDQJKJakPAuUxazIXcBww2Qfprp7ExzRz-VwG7j4Y1D09_IcmpZX8iJGkCxT7UVyHsQesWLHuraDMuNTMHpKJfA2sQJ0733ZC4q5C-XXUJGBXu0mxEex3mxchAjsm1jw0jxjAWlSvpm4pfNGkGtnBwXQuwyJHC6kbMxIRSsKiKjAhJ91iyB6TmpPRfu6WPlKYq1lsimbhBh-WVyV3TFkalEpQiALshnFJ_t6qMdQpROXg67MVGG1mfPRhDisU3oI4k6WYK_phFzz_ENAGh25ODMKpeasskOvh91R16dFnciVlNk87_ImM0Nn5lPYtK6--w; vcloud_session_id=9e637cb489b94c1c9a3bd3e7fa402c4b; _pk_ses.1.928c=0; _pk_id.1.928c=b42eee8f5b04b65f.1650899400.12.1652783099.1652782931..e167dd23fb28b446bfcec22138c190a5805e5757b2109895657d6b5975b97d4e', 'Accept': 'application/+json;version=36.0, application/json;version=36.0', 'Connection': 'keep-alive', 'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.54 Safari/537.36', 'Referer': 'https://vcd.dc.schwunk-it.de/provider/plugins/Vk13YXJl/cse', 'Sec-Fetch-Site': 'same-origin', 'Sec-Fetch-Dest': 'empty', 'Host': 'vcd.dc.schwunk-it.de', 'Accept-Encoding': 'gzip, deflate, br', 'Sec-Fetch-Mode': 'cors', 'Authorization': '[REDACTED]', 'sec-ch-ua': '" Not A;Brand";v="99", "Chromium";v="101", "Google Chrome";v="101"', 'sec-ch-ua-mobile': '?0', 'X-VMWARE-VCLOUD-TENANT-CONTEXT': '6d101c73-a61a-4173-b1e4-8e5569a9055b', 'sec-ch-ua-platform': '"macOS"', 'Accept-Language': 'en-US,en;q=0.9,de;q=0.8,de-DE;q=0.7', 'Content-Length': '925', 'Content-Type': 'application/+json'}, 'body': 'eyJhcGlWZXJzaW9uIjoiY3NlLnZtd2FyZS5jb20vdjIuMSIsImtpbmQiOiJUS0dtIiwibWV0YWRhdGEiOnsiYWRkaXRpb25hbFByb3BlcnRpZXMiOnRydWUsIm9yZ05hbWUiOiJEZW1vIiwidmlydHVhbERhdGFDZW50ZXJOYW1lIjoiRGVtby1WREMiLCJuYW1lIjoiazhzIiwic2l0ZSI6Imh0dHBzOi8vdmNkLmRjLnNjaHd1bmstaXQuZGUifSwic3BlYyI6eyJhZGRpdGlvbmFsUHJvcGVydGllcyI6dHJ1ZSwidG9wb2xvZ3kiOnsiY29udHJvbFBsYW5lIjp7ImNvdW50IjoxLCJzaXppbmdDbGFzcyI6Ims4cyIsInN0b3JhZ2VQcm9maWxlIjoidGFuenUiLCJjcHUiOm51bGwsIm1lbW9yeSI6bnVsbH0sIndvcmtlcnMiOnsiY291bnQiOjIsInNpemluZ0NsYXNzIjoiazhzIiwic3RvcmFnZVByb2ZpbGUiOiJ0YW56dSIsImNwdSI6bnVsbCwibWVtb3J5IjpudWxsfSwibmZzIjp7ImNvdW50IjowLCJzaXppbmdDbGFzcyI6bnVsbCwic3RvcmFnZVByb2ZpbGUiOm51bGx9fSwic2V0dGluZ3MiOnsib3ZkY05ldHdvcmsiOiJEZW1vIiwicm9sbGJhY2tPbkZhaWx1cmUiOnRydWUsIm5ldHdvcmsiOnsiZXhwb3NlIjp0cnVlLCJwb2RzIjp7ImNpZHJCbG9ja3MiOlsiMTAwLjk2LjAuMC8xMSJdfSwic2VydmljZXMiOnsiY2lkckJsb2NrcyI6WyIxMDAuNjQuMC4wLzEzIl19fSwiY3NpIjpbeyJkZWZhdWx0SzhzU3RvcmFnZUNsYXNzIjp7InZjZFN0b3JhZ2VQcm9maWxlTmFtZSI6InRhbnp1IiwiazhzU3RvcmFnZUNsYXNzTmFtZSI6InRhbnp1IiwidXNlRGVsZXRlUmVjbGFpbVBvbGljeSI6dHJ1ZSwiZmlsZXN5c3RlbSI6ImV4dDQifX1dfSwiZGlzdHJpYnV0aW9uIjp7InRlbXBsYXRlTmFtZSI6InVidW50dS0yMDA0LWt1YmUtdjEuMjEuMit2bXdhcmUuMS10a2cuMS03ODMyOTA3NzkxOTg0NDk4MzIyIiwidGVtcGxhdGVSZXZpc2lvbiI6MX19fQ==', 'statusCode': 0, 'request': True} 22-05-17 10:25:00 | mqtt_publisher:116 - send_response | Request Id: fe8f8703-833e-4033-b50c-94d0d15d297d | DEBUG :: publish return (rc, msg_id): (0, 6) 22-05-17 10:25:00 | mqtt_consumer:85 - process_mqtt_message | Request Id: fe8f8703-833e-4033-b50c-94d0d15d297d | DEBUG :: MQTT response: {'type': 'API_RESPONSE', 'headers': {'requestId': 'fe8f8703-833e-4033-b50c-94d0d15d297d'}, 'httpResponse': {'statusCode': 500, 'headers': {'Content-Type': 'application/json', 'Content-Length': 128}, 'body': 'eyJtZXNzYWdlIjogeyJtaW5vciBlcnJvciBjb2RlIjogLTEsICJlcnJvciBkZXNjcmlwdGlvbiI6ICJbIGFkZmE4NThhLWIzNjgtNDEwZS1iNGI2LTA4MGZkMzIyYTQ5MyBdIFRoaXMgb3BlcmF0aW9uIGlzIGRlbmllZC4ifX0='}} 22-05-17 10:25:31 | request_dispatcher:894 - process_request | Request Id: 0bf15af1-2fca-41f5-a8bb-4028f28400eb | DEBUG :: Incoming request message: {"id": "96750abd-5e72-4d72-a80b-5856ae372111", "method": "POST", "requestUri": "/api/cse/3.0/clusters", "queryString": null, "protocol": "HTTP/1.1", "scheme": "https", "remoteAddr": "192.168.254.1", "remotePort": 52479, "localAddr": "192.168.254.31", "localPort": 443, "headers": {"Origin": "https://vcd.dc.schwunk-it.de", "Cookie": "_ga=GA1.2.130519610.1639845461; vcloud_jwt=eyJhbGciOiJSUzI1NiJ9.eyJzdWIiOiJhZG1pbmlzdHJhdG9yIiwiaXNzIjoiYTkzYzlkYjktNzQ3MS0zMTkyLThkMDktYThmN2VlZGE4NWY5QDAzM2UzMjJmLWZkZGEtNDJkOS1iOWJmLTM5MjI3MzNkOTMxNyIsImV4cCI6MTY1Mjg2OTI3MiwidmVyc2lvbiI6InZjbG91ZF8xLjAiLCJqdGkiOiI5ZTYzN2NiNDg5Yjk0YzFjOWEzYmQzZTdmYTQwMmM0YiJ9.FvUJhmWnJ2p7u8nlq7lfjDQJKJakPAuUxazIXcBww2Qfprp7ExzRz-VwG7j4Y1D09_IcmpZX8iJGkCxT7UVyHsQesWLHuraDMuNTMHpKJfA2sQJ0733ZC4q5C-XXUJGBXu0mxEex3mxchAjsm1jw0jxjAWlSvpm4pfNGkGtnBwXQuwyJHC6kbMxIRSsKiKjAhJ91iyB6TmpPRfu6WPlKYq1lsimbhBh-WVyV3TFkalEpQiALshnFJ_t6qMdQpROXg67MVGG1mfPRhDisU3oI4k6WYK_phFzz_ENAGh25ODMKpeasskOvh91R16dFnciVlNk87_ImM0Nn5lPYtK6--w; vcloud_session_id=9e637cb489b94c1c9a3bd3e7fa402c4b; _pk_ses.1.928c=0; _pk_id.1.928c=b42eee8f5b04b65f.1650899400.12.1652783131.1652782931..e167dd23fb28b446bfcec22138c190a5805e5757b2109895657d6b5975b97d4e", "Accept": "application/+json;version=36.0, application/json;version=36.0", "Connection": "keep-alive", "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.54 Safari/537.36", "Referer": "https://vcd.dc.schwunk-it.de/provider/plugins/Vk13YXJl/cse", "Sec-Fetch-Site": "same-origin", "Sec-Fetch-Dest": "empty", "Host": "vcd.dc.schwunk-it.de", "Accept-Encoding": "gzip, deflate, br", "Sec-Fetch-Mode": "cors", "Authorization": [[REDACTED], "sec-ch-ua": "\" Not A;Brand\";v=\"99\", \"Chromium\";v=\"101\", \"Google Chrome\";v=\"101\"", "sec-ch-ua-mobile": "?0", "X-VMWARE-VCLOUD-TENANT-CONTEXT": "6d101c73-a61a-4173-b1e4-8e5569a9055b", "sec-ch-ua-platform": "\"macOS\"", "Accept-Language": "en-US,en;q=0.9,de;q=0.8,de-DE;q=0.7", "Content-Length": "925", "Content-Type": "application/+json"}, "body": "eyJhcGlWZXJzaW9uIjoiY3NlLnZtd2FyZS5jb20vdjIuMSIsImtpbmQiOiJUS0dtIiwibWV0YWRhdGEiOnsiYWRkaXRpb25hbFByb3BlcnRpZXMiOnRydWUsIm9yZ05hbWUiOiJEZW1vIiwidmlydHVhbERhdGFDZW50ZXJOYW1lIjoiRGVtby1WREMiLCJuYW1lIjoiazhzIiwic2l0ZSI6Imh0dHBzOi8vdmNkLmRjLnNjaHd1bmstaXQuZGUifSwic3BlYyI6eyJhZGRpdGlvbmFsUHJvcGVydGllcyI6dHJ1ZSwidG9wb2xvZ3kiOnsiY29udHJvbFBsYW5lIjp7ImNvdW50IjoxLCJzaXppbmdDbGFzcyI6Ims4cyIsInN0b3JhZ2VQcm9maWxlIjoidGFuenUiLCJjcHUiOm51bGwsIm1lbW9yeSI6bnVsbH0sIndvcmtlcnMiOnsiY291bnQiOjIsInNpemluZ0NsYXNzIjoiazhzIiwic3RvcmFnZVByb2ZpbGUiOiJ0YW56dSIsImNwdSI6bnVsbCwibWVtb3J5IjpudWxsfSwibmZzIjp7ImNvdW50IjowLCJzaXppbmdDbGFzcyI6bnVsbCwic3RvcmFnZVByb2ZpbGUiOm51bGx9fSwic2V0dGluZ3MiOnsib3ZkY05ldHdvcmsiOiJEZW1vIiwicm9sbGJhY2tPbkZhaWx1cmUiOnRydWUsIm5ldHdvcmsiOnsiZXhwb3NlIjp0cnVlLCJwb2RzIjp7ImNpZHJCbG9ja3MiOlsiMTAwLjk2LjAuMC8xMSJdfSwic2VydmljZXMiOnsiY2lkckJsb2NrcyI6WyIxMDAuNjQuMC4wLzEzIl19fSwiY3NpIjpbeyJkZWZhdWx0SzhzU3RvcmFnZUNsYXNzIjp7InZjZFN0b3JhZ2VQcm9maWxlTmFtZSI6InRhbnp1IiwiazhzU3RvcmFnZUNsYXNzTmFtZSI6InRhbnp1IiwidXNlRGVsZXRlUmVjbGFpbVBvbGljeSI6dHJ1ZSwiZmlsZXN5c3RlbSI6ImV4dDQifX1dfSwiZGlzdHJpYnV0aW9uIjp7InRlbXBsYXRlTmFtZSI6InVidW50dS0yMDA0LWt1YmUtdjEuMjEuMit2bXdhcmUuMS10a2cuMS03ODMyOTA3NzkxOTg0NDk4MzIyIiwidGVtcGxhdGVSZXZpc2lvbiI6MX19fQ==", "statusCode": 0, "request": true} 22-05-17 10:25:31 | request_dispatcher:1020 - process_request | Request Id: 0bf15af1-2fca-41f5-a8bb-4028f28400eb | DEBUG :: request body: {'apiVersion': 'cse.vmware.com/v2.1', 'kind': 'TKGm', 'metadata': {'additionalProperties': True, 'orgName': 'Demo', 'virtualDataCenterName': 'Demo-VDC', 'name': 'k8s', 'site': 'https://vcd.dc.schwunk-it.de'}, 'spec': {'additionalProperties': True, 'topology': {'controlPlane': {'count': 1, 'sizingClass': 'k8s', 'storageProfile': 'tanzu', 'cpu': None, 'memory': None}, 'workers': {'count': 2, 'sizingClass': 'k8s', 'storageProfile': 'tanzu', 'cpu': None, 'memory': None}, 'nfs': {'count': 0, 'sizingClass': None, 'storageProfile': None}}, 'settings': {'ovdcNetwork': 'Demo', 'rollbackOnFailure': True, 'network': {'expose': True, 'pods': {'cidrBlocks': ['100.96.0.0/11']}, 'services': {'cidrBlocks': ['100.64.0.0/13']}}, 'csi': [{'defaultK8sStorageClass': {'vcdStorageProfileName': 'tanzu', 'k8sStorageClassName': 'tanzu', 'useDeleteReclaimPolicy': True, 'filesystem': 'ext4'}}]}, 'distribution': {'templateName': 'ubuntu-2004-kube-v1.21.2+vmware.1-tkg.1-7832907791984498322', 'templateRevision': 1}}} 22-05-17 10:25:32 | entity_service:56 - exception_handler_wrapper | Request Id: 0bf15af1-2fca-41f5-a8bb-4028f28400eb | ERROR :: [ 4eb1f2d2-bd4a-4d52-b4ce-010c6bddf2ea ] This operation is denied. Traceback (most recent call last): File "/opt/cse/python/lib/python3.7/site-packages/container_service_extension/rde/common/entity_service.py", line 52, in exception_handler_wrapper result = func(*args, *kwargs) File "/opt/cse/python/lib/python3.7/site-packages/container_service_extension/rde/common/entity_service.py", line 113, in create_entity return_response_headers=is_request_async) File "/opt/cse/python/lib/python3.7/site-packages/container_service_extension/lib/cloudapi/cloudapi_client.py", line 140, in do_request response.raise_for_status() File "/opt/cse/python/lib/python3.7/site-packages/requests/models.py", line 960, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://vcd.dc.schwunk-it.de/cloudapi/1.0.0/entityTypes/urn:vcloud:type:cse:nativeCluster:2.1.0 22-05-17 10:25:32 | request_utils:166 - exception_handler_wrapper | Request Id: 0bf15af1-2fca-41f5-a8bb-4028f28400eb | ERROR :: [ 4eb1f2d2-bd4a-4d52-b4ce-010c6bddf2ea ] This operation is denied. 22-05-17 10:25:32 | exception_handler:53 - exception_handler_wrapper | Request Id: 0bf15af1-2fca-41f5-a8bb-4028f28400eb | ERROR :: Traceback (most recent call last): File "/opt/cse/python/lib/python3.7/site-packages/container_service_extension/rde/common/entity_service.py", line 52, in exception_handler_wrapper result = func(args, **kwargs) File "/opt/cse/python/lib/python3.7/site-packages/container_service_extension/rde/common/entity_service.py", line 113, in create_entity return_response_headers=is_request_async) File "/opt/cse/python/lib/python3.7/site-packages/container_service_extension/lib/cloudapi/cloudapi_client.py", line 140, in do_request response.raise_for_status() File "/opt/cse/python/lib/python3.7/site-packages/requests/models.py", line 960, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://vcd.dc.schwunk-it.de/cloudapi/1.0.0/entityTypes/urn:vcloud:type:cse:nativeCluster:2.1.0

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/opt/cse/python/lib/python3.7/site-packages/container_service_extension/exception/exception_handler.py", line 37, in exception_handler_wrapper result = func(*args, kwargs) File "/opt/cse/python/lib/python3.7/site-packages/container_service_extension/server/request_dispatcher.py", line 1044, in process_request body_content = handler_method(request_data, operation_ctx) File "/opt/cse/python/lib/python3.7/site-packages/container_service_extension/lib/telemetry/telemetry_handler.py", line 113, in wrapper raise err File "/opt/cse/python/lib/python3.7/site-packages/container_service_extension/lib/telemetry/telemetry_handler.py", line 107, in wrapper ret_value = func(*args, *kwargs) File "/opt/cse/python/lib/python3.7/site-packages/container_service_extension/server/request_handlers/request_utils.py", line 167, in exception_handler_wrapper raise error File "/opt/cse/python/lib/python3.7/site-packages/container_service_extension/server/request_handlers/request_utils.py", line 161, in exception_handler_wrapper result = func(args, kwargs) File "/opt/cse/python/lib/python3.7/site-packages/container_service_extension/server/request_handlers/cluster_handler.py", line 87, in cluster_create is_request_async=True) File "/opt/cse/python/lib/python3.7/site-packages/container_service_extension/rde/common/entity_service.py", line 59, in exception_handler_wrapper minor_error_code=MinorErrorCode.DEFAULT_ERROR_CODE) container_service_extension.exception.exceptions.DefEntityServiceError: [ 4eb1f2d2-bd4a-4d52-b4ce-010c6bddf2ea ] This operation is denied.

22-05-17 10:25:32 | mqtt_consumer:73 - process_mqtt_message | Request Id: 0bf15af1-2fca-41f5-a8bb-4028f28400eb | DEBUG :: Received message with request_id: 0bf15af1-2fca-41f5-a8bb-4028f28400eb, mid: 6, and msg json: {'id': '96750abd-5e72-4d72-a80b-5856ae372111', 'method': 'POST', 'requestUri': '/api/cse/3.0/clusters', 'queryString': None, 'protocol': 'HTTP/1.1', 'scheme': 'https', 'remoteAddr': '192.168.254.1', 'remotePort': 52479, 'localAddr': '192.168.254.31', 'localPort': 443, 'headers': {'Origin': 'https://vcd.dc.schwunk-it.de', 'Cookie': '_ga=GA1.2.130519610.1639845461; vcloud_jwt=eyJhbGciOiJSUzI1NiJ9.eyJzdWIiOiJhZG1pbmlzdHJhdG9yIiwiaXNzIjoiYTkzYzlkYjktNzQ3MS0zMTkyLThkMDktYThmN2VlZGE4NWY5QDAzM2UzMjJmLWZkZGEtNDJkOS1iOWJmLTM5MjI3MzNkOTMxNyIsImV4cCI6MTY1Mjg2OTI3MiwidmVyc2lvbiI6InZjbG91ZF8xLjAiLCJqdGkiOiI5ZTYzN2NiNDg5Yjk0YzFjOWEzYmQzZTdmYTQwMmM0YiJ9.FvUJhmWnJ2p7u8nlq7lfjDQJKJakPAuUxazIXcBww2Qfprp7ExzRz-VwG7j4Y1D09_IcmpZX8iJGkCxT7UVyHsQesWLHuraDMuNTMHpKJfA2sQJ0733ZC4q5C-XXUJGBXu0mxEex3mxchAjsm1jw0jxjAWlSvpm4pfNGkGtnBwXQuwyJHC6kbMxIRSsKiKjAhJ91iyB6TmpPRfu6WPlKYq1lsimbhBh-WVyV3TFkalEpQiALshnFJ_t6qMdQpROXg67MVGG1mfPRhDisU3oI4k6WYK_phFzz_ENAGh25ODMKpeasskOvh91R16dFnciVlNk87_ImM0Nn5lPYtK6--w; vcloud_session_id=9e637cb489b94c1c9a3bd3e7fa402c4b; _pk_ses.1.928c=0; _pk_id.1.928c=b42eee8f5b04b65f.1650899400.12.1652783131.1652782931..e167dd23fb28b446bfcec22138c190a5805e5757b2109895657d6b5975b97d4e', 'Accept': 'application/+json;version=36.0, application/json;version=36.0', 'Connection': 'keep-alive', 'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.54 Safari/537.36', 'Referer': 'https://vcd.dc.schwunk-it.de/provider/plugins/Vk13YXJl/cse', 'Sec-Fetch-Site': 'same-origin', 'Sec-Fetch-Dest': 'empty', 'Host': 'vcd.dc.schwunk-it.de', 'Accept-Encoding': 'gzip, deflate, br', 'Sec-Fetch-Mode': 'cors', 'Authorization': '[REDACTED]', 'sec-ch-ua': '" Not A;Brand";v="99", "Chromium";v="101", "Google Chrome";v="101"', 'sec-ch-ua-mobile': '?0', 'X-VMWARE-VCLOUD-TENANT-CONTEXT': '6d101c73-a61a-4173-b1e4-8e5569a9055b', 'sec-ch-ua-platform': '"macOS"', 'Accept-Language': 'en-US,en;q=0.9,de;q=0.8,de-DE;q=0.7', 'Content-Length': '925', 'Content-Type': 'application/+json'}, 'body': 'eyJhcGlWZXJzaW9uIjoiY3NlLnZtd2FyZS5jb20vdjIuMSIsImtpbmQiOiJUS0dtIiwibWV0YWRhdGEiOnsiYWRkaXRpb25hbFByb3BlcnRpZXMiOnRydWUsIm9yZ05hbWUiOiJEZW1vIiwidmlydHVhbERhdGFDZW50ZXJOYW1lIjoiRGVtby1WREMiLCJuYW1lIjoiazhzIiwic2l0ZSI6Imh0dHBzOi8vdmNkLmRjLnNjaHd1bmstaXQuZGUifSwic3BlYyI6eyJhZGRpdGlvbmFsUHJvcGVydGllcyI6dHJ1ZSwidG9wb2xvZ3kiOnsiY29udHJvbFBsYW5lIjp7ImNvdW50IjoxLCJzaXppbmdDbGFzcyI6Ims4cyIsInN0b3JhZ2VQcm9maWxlIjoidGFuenUiLCJjcHUiOm51bGwsIm1lbW9yeSI6bnVsbH0sIndvcmtlcnMiOnsiY291bnQiOjIsInNpemluZ0NsYXNzIjoiazhzIiwic3RvcmFnZVByb2ZpbGUiOiJ0YW56dSIsImNwdSI6bnVsbCwibWVtb3J5IjpudWxsfSwibmZzIjp7ImNvdW50IjowLCJzaXppbmdDbGFzcyI6bnVsbCwic3RvcmFnZVByb2ZpbGUiOm51bGx9fSwic2V0dGluZ3MiOnsib3ZkY05ldHdvcmsiOiJEZW1vIiwicm9sbGJhY2tPbkZhaWx1cmUiOnRydWUsIm5ldHdvcmsiOnsiZXhwb3NlIjp0cnVlLCJwb2RzIjp7ImNpZHJCbG9ja3MiOlsiMTAwLjk2LjAuMC8xMSJdfSwic2VydmljZXMiOnsiY2lkckJsb2NrcyI6WyIxMDAuNjQuMC4wLzEzIl19fSwiY3NpIjpbeyJkZWZhdWx0SzhzU3RvcmFnZUNsYXNzIjp7InZjZFN0b3JhZ2VQcm9maWxlTmFtZSI6InRhbnp1IiwiazhzU3RvcmFnZUNsYXNzTmFtZSI6InRhbnp1IiwidXNlRGVsZXRlUmVjbGFpbVBvbGljeSI6dHJ1ZSwiZmlsZXN5c3RlbSI6ImV4dDQifX1dfSwiZGlzdHJpYnV0aW9uIjp7InRlbXBsYXRlTmFtZSI6InVidW50dS0yMDA0LWt1YmUtdjEuMjEuMit2bXdhcmUuMS10a2cuMS03ODMyOTA3NzkxOTg0NDk4MzIyIiwidGVtcGxhdGVSZXZpc2lvbiI6MX19fQ==', 'statusCode': 0, 'request': True} 22-05-17 10:25:32 | mqtt_publisher:116 - send_response | Request Id: 0bf15af1-2fca-41f5-a8bb-4028f28400eb | DEBUG :: publish return (rc, msg_id): (0, 7) 22-05-17 10:25:32 | mqtt_consumer:85 - process_mqtt_message | Request Id: 0bf15af1-2fca-41f5-a8bb-4028f28400eb | DEBUG :: MQTT response: {'type': 'API_RESPONSE', 'headers': {'requestId': '0bf15af1-2fca-41f5-a8bb-4028f28400eb'}, 'httpResponse': {'statusCode': 500, 'headers': {'Content-Type': 'application/json', 'Content-Length': 128}, 'body': 'eyJtZXNzYWdlIjogeyJtaW5vciBlcnJvciBjb2RlIjogLTEsICJlcnJvciBkZXNjcmlwdGlvbiI6ICJbIDRlYjFmMmQyLWJkNGEtNGQ1Mi1iNGNlLTAxMGM2YmRkZjJlYSBdIFRoaXMgb3BlcmF0aW9uIGlzIGRlbmllZC4ifX0='}}

Cloud Director 10.3.3 CSE 3.1.3

Reproduction steps

1. Open VCD and create a new Tanzu Kubernetes Cluster
2. Click Finish
3.
...

Expected behavior

?

Additional context

No response

sahithi commented 2 years ago 
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://vcd.dc.schwunk-it.de/cloudapi/1.0.0/entityTypes/urn:vcloud:type:cse:nativeCluster:2.1.0

This means the user does not have cse:nativeCluster rights. Please assign cse:nativeCluster Full control rights and retry the operation