chaostoolkit-incubator / chaostoolkit-azure

Chaos Toolkit Extension for Azure
https://chaostoolkit.org/
Apache License 2.0
22 stars 28 forks source link

Azure China Cloud setting does not work #136

Closed xpdable closed 2 years ago

xpdable commented 2 years ago

I implemented this feature years ago with a setting to specify the cloud:

    "secrets": {
        "azure": {
                "client_id": "",
                "client_secret": "",
                "tenant_id": "",
                "azure_cloud": "AZURE_CHINA_CLOUD",
        }
    },

Now it is not working with latest code

{
  "chaoslib-version": "1.28.0",
  "platform": "macOS-12.4-x86_64-i386-64bit",
  "node": "xps-MacBook-Pro.local",
  "experiment": {
    "title": "Stop VM",
    "description": "Stop VM in Azure",
    "method": [
      {
        "type": "action",
        "name": "stop-vm",
        "provider": {
          "type": "python",
          "module": "chaosazure.machine.actions",
          "func": "stop_machines",
          "arguments": {
            "filter": "where tolower(resourceGroup) == 'iapimcninfresgp004' and tolower(name) == 'IAPIMCNINFVMSA004'"
          },
          "secrets": [
            "azure"
          ],
          "configuration": [
            "azure"
          ]
        }
      }
    ],
    "secrets": {
      "azure": {
        "client_id": "***",
        "client_secret": ".***-",
        "tenant_id": "***",
        "azure_cloud": "AZURE_CHINA_CLOUD"
      }
    },
    "configuration": {
      "azure": {
        "subscription_id": "***"
      },
      "environment": "azure"
    },
    "dry": null
  },
  "start": "2022-07-06T09:04:40.466148",
  "status": "completed",
  "deviated": false,
  "steady_states": {
    "before": null,
    "after": null,
    "during": []
  },
  "run": [
    {
      "activity": {
        "type": "action",
        "name": "stop-vm",
        "provider": {
          "type": "python",
          "module": "chaosazure.machine.actions",
          "func": "stop_machines",
          "arguments": {
            "filter": "where tolower(resourceGroup) == 'iapimcninfresgp004' and tolower(name) == 'IAPIMCNINFVMSA004'"
          },
          "secrets": [
            "azure"
          ],
          "configuration": [
            "azure"
          ]
        }
      },
      "output": null,
      "status": "failed",
      "exception": [
        "Traceback (most recent call last):\n",
        "  File \"/usr/local/lib/python3.9/site-packages/chaosazure/common/resources/graph.py\", line 22, in fetch_resources\n    resources = client.resources(_query_request)\n",
        "  File \"/usr/local/lib/python3.9/site-packages/azure/mgmt/resourcegraph/operations/_resource_graph_client_operations.py\", line 67, in resources\n    pipeline_response = self._client._pipeline.run(request, stream=False, **kwargs)\n",
        "  File \"/usr/local/lib/python3.9/site-packages/azure/core/pipeline/_base.py\", line 211, in run\n    return first_node.send(pipeline_request)  # type: ignore\n",
        "  File \"/usr/local/lib/python3.9/site-packages/azure/core/pipeline/_base.py\", line 71, in send\n    response = self.next.send(request)\n",
        "  File \"/usr/local/lib/python3.9/site-packages/azure/core/pipeline/_base.py\", line 71, in send\n    response = self.next.send(request)\n",
        "  File \"/usr/local/lib/python3.9/site-packages/azure/core/pipeline/_base.py\", line 71, in send\n    response = self.next.send(request)\n",
        "  [Previous line repeated 2 more times]\n",
        "  File \"/usr/local/lib/python3.9/site-packages/azure/mgmt/core/policies/_base.py\", line 47, in send\n    response = self.next.send(request)\n",
        "  File \"/usr/local/lib/python3.9/site-packages/azure/core/pipeline/policies/_redirect.py\", line 158, in send\n    response = self.next.send(request)\n",
        "  File \"/usr/local/lib/python3.9/site-packages/azure/core/pipeline/policies/_retry.py\", line 446, in send\n    response = self.next.send(request)\n",
        "  File \"/usr/local/lib/python3.9/site-packages/azure/core/pipeline/policies/_authentication.py\", line 117, in send\n    self.on_request(request)\n",
        "  File \"/usr/local/lib/python3.9/site-packages/azure/core/pipeline/policies/_authentication.py\", line 94, in on_request\n    self._token = self._credential.get_token(*self._scopes)\n",
        "  File \"/usr/local/lib/python3.9/site-packages/azure/identity/_internal/get_token_mixin.py\", line 76, in get_token\n    token = self._request_token(*scopes, **kwargs)\n",
        "  File \"/usr/local/lib/python3.9/site-packages/azure/identity/_internal/decorators.py\", line 74, in wrapper\n    return fn(*args, **kwargs)\n",
        "  File \"/usr/local/lib/python3.9/site-packages/azure/identity/_internal/client_credential_base.py\", line 40, in _request_token\n    raise ClientAuthenticationError(message=message)\n",
        "azure.core.exceptions.ClientAuthenticationError: Authentication failed: AADSTS500011: The resource principal named https://management.azure.com was not found in the tenant named Mercedes-Benz (corpdir.partner.onmschina.cn). This can happen if the application has not been installed by the administrator of the tenant or consented to by any user in the tenant. You might have sent your authentication request to the wrong tenant.\r\nTrace ID: c794a89f-639c-4029-86b0-b27f61b35202\r\nCorrelation ID: 09ec80d0-e01d-406b-9b44-4d19c0737bc8\r\nTimestamp: 2022-07-06 09:04:42Z\n",
        "\nDuring handling of the above exception, another exception occurred:\n\n",
        "Traceback (most recent call last):\n",
        "  File \"/usr/local/lib/python3.9/site-packages/chaoslib/provider/python.py\", line 56, in run_python_activity\n    return func(**arguments)\n",
        "  File \"/usr/local/lib/python3.9/site-packages/chaosazure/machine/actions.py\", line 96, in stop_machines\n    machines = __fetch_machines(filter, configuration, secrets)\n",
        "  File \"/usr/local/lib/python3.9/site-packages/chaosazure/machine/actions.py\", line 512, in __fetch_machines\n    machines = fetch_resources(filter, RES_TYPE_VM, secrets, configuration)\n",
        "  File \"/usr/local/lib/python3.9/site-packages/chaosazure/common/resources/graph.py\", line 24, in fetch_resources\n    msg = e.error.code\n",
        "AttributeError: 'NoneType' object has no attribute 'code'\n",
        "\nDuring handling of the above exception, another exception occurred:\n\n",
        "chaoslib.exceptions.ActivityFailed: AttributeError: 'NoneType' object has no attribute 'code'\n"
      ],
      "start": "2022-07-06T09:04:40.467684",
      "end": "2022-07-06T09:04:42.395064",
      "duration": 1.92738
    }
  ],
  "rollbacks": [],
  "end": "2022-07-06T09:04:42.399450",
  "duration": 1.9333240985870361
}%
xpdable commented 2 years ago

I even hardcode the base url in the init py to :

def init_resource_graph_client(
        experiment_secrets: Secrets) -> ResourceGraphClient:
    """
    Initializes Resource Graph client.
    """
    secrets = load_secrets(experiment_secrets)
    with auth(secrets) as authentication:
        base_url = secrets.get('cloud').endpoints.resource_manager
        client = ResourceGraphClient(
            credential=authentication,
            base_url="https://management.chinacloudapi.cn")

        return client
xpdable commented 2 years ago

OK, I find sth. The current code is using ClientSecretCredentialBase from Azure Python SDK My commit which works used from msrestazure.azure_active_directory import ServicePrincipalCredentials which works with the configure of from msrestazure.azure_cloud import AZURE_PUBLIC_CLOUD, AZURE_US_GOV_CLOUD, AZURE_GERMAN_CLOUD, AZURE_CHINA_CLOUD this is not work with Azure Python SDK

@russmiles @Lawouach @buderre Any suggestion? I will have to modify my previous code maybe

xpdable commented 2 years ago

Seems not a issue with CTK. reproduced just by scripts. I've raised support case to Azure.

xpdable commented 2 years ago

Seems not a issue with CTK. reproduced just by scripts. I've raised support case to Azure.

Seems Azure support is not so professional as me, still waiting for the solution

xpdable commented 2 years ago

Ok, it is provide that *Configuration of Client of Azure SDK now accepts credential_scopes to eventually do the conection. You will specify in the Client init

    client = ComputeManagementClient(
        credential=result,
        client_credential=result,
        credential_scopes=('https://management.chinacloudapi.cn/.default',),
        subscription_id='ed886107-0f48-4043-835a-2394a94f21a8',
        base_url=base_url)
class ComputeManagementClientConfiguration(Configuration):
...
    self.credential_scopes = kwargs.pop('credential_scopes', ['https://management.azure.com/.default'])
xpdable commented 2 years ago

More words that Azure support sucks