ansible-collections / azure

Development area for Azure Collections
https://galaxy.ansible.com/azure/azcollection
GNU General Public License v3.0
245 stars 327 forks source link

Azure Account Info Not Populating #1464

Closed thisguyshouldworkforus closed 5 months ago

thisguyshouldworkforus commented 6 months ago
SUMMARY

We have created a Service Principal in Azure, and validated those credentials with AzureCLI az login --service-principal --username='12345678-abcd-1234-efgh-567890abcdef' --password='SuperSecretValue1234!' --tenant='98765432-abcd-1234-cdef-1234567890ab' -- This works as expected, and we see the account data we expect with az account show.

We have loaded these credentials into the Ansible Automation Platform (AAP) and have updated the playbook to pull these values from the environment:

azure.azcollection.azure_rm_account_info:
  auth_source: env
  client_id: "{{ lookup('env', 'AZURE_CLIENT_ID') }}"
  cloud_environment: "{{ lookup('env', 'AZURE_CLOUD_ENVIRONMENT') }}"
  secret: "{{ lookup('env', 'AZURE_SECRET') }}"
  subscription_id: "{{ lookup('env', 'AZURE_SUBSCRIPTION_ID') }}"
  tenant: "{{ lookup('env', 'AZURE_TENANT') }}"

We also first listed the environment variables with a debug statement to verify/validate that the proper information was infact being passed to the playbook ... it is all correct.

ISSUE TYPE
COMPONENT NAME
azure.azcollection.azure_rm_account_info
ANSIBLE VERSION (Execution Environment[ee-supported-rhel8])
ansible-core, 2.15.9
COLLECTION VERSION (Execution Environment[ee-supported-rhel8])
---
ansible:
  collections:
    details:
      amazon.aws: 6.4.0
      ansible.controller: 4.5.1
      ansible.netcommon: 6.0.0
      ansible.network: 3.0.0
      ansible.posix: 1.5.4
      ansible.scm: 2.0.0
      ansible.security: 2.0.0
      ansible.snmp: 2.0.0
      ansible.utils: 3.0.0
      ansible.windows: 1.14.0
      ansible.yang: 2.0.0
      arista.eos: 7.0.0
      azure.azcollection: 2.2.0
      cisco.asa: 5.0.0
      cisco.ios: 6.1.0
      cisco.iosxr: 7.0.0
      cisco.nxos: 6.0.0
      cloud.common: 2.1.2
      cloud.terraform: 1.1.1
      community.general: 8.3.0
      frr.frr: 2.0.2
      ibm.qradar: 3.0.0
      junipernetworks.junos: 6.0.0
      kubernetes.core: 2.4.0
      microsoft.ad: 1.1.0
      openvswitch.openvswitch: 2.1.1
      redhat.amq_broker: 1.3.0
      redhat.eap: 1.3.1
      redhat.insights: 1.0.7
      redhat.openshift: 2.3.0
      redhat.redhat_csp_download: 1.2.2
      redhat.rhel_idm: 1.10.0
      redhat.rhel_system_roles: 1.21.1
      redhat.rhv: 2.4.2
      redhat.runtimes_common: 1.0.2
      redhat.sap_install: 1.2.1
      redhat.satellite: 3.10.0
      redhat.satellite_operations: 1.3.0
      redhat.sso: 1.2.1
      sap.sap_operations: 1.0.4
      servicenow.itsm: 2.1.0
      splunk.es: 3.0.0
      trendmicro.deepsec: 3.0.0
      vmware.vmware_rest: 2.3.1
      vyos.vyos: 4.0.2
  version:
    details: ansible [core 2.15.9]
CONFIGURATION (Execution Environment[ee-supported-rhel8])
ansible-config dump --only-changed
CONFIG_FILE() = /etc/ansible/ansible.cfg
OS / ENVIRONMENT (Execution Environment[ee-supported-rhel8])
---
friendly:
  details: |-
    Red Hat Enterprise Linux release 8.9 (Ootpa)
os:
  details:
  - ansi-color: 0;31
    bug-report-url: 'https://bugzilla.redhat.com/ '
    cpe-name: cpe:/o:redhat:enterprise_linux:8::baseos
    documentation-url: https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8
    home-url: https://www.redhat.com/
    id: rhel
    id-like: fedora
    name: Red Hat Enterprise Linux
    platform-id: platform:el8
    pretty-name: Red Hat Enterprise Linux 8.9 (Ootpa)
    redhat-bugzilla-product: Red Hat Enterprise Linux 8
    redhat-bugzilla-product-version: '8.9'
    redhat-support-product: Red Hat Enterprise Linux
    redhat-support-product-version: '8.9'
    version: 8.9 (Ootpa)
    version-id: '8.9'
python:
  details:
    version: 3.9.18 (main, Sep 22 2023, 17:58:34)  [GCC 8.5.0 20210514 (Red Hat 8.5.0-20)]
STEPS TO REPRODUCE
  azure.azcollection.azure_rm_account_info:
    auth_source: env
    client_id: "{{ lookup('env', 'AZURE_CLIENT_ID') }}"
    cloud_environment: "{{ lookup('env', 'AZURE_CLOUD_ENVIRONMENT') }}"
    secret: "{{ lookup('env', 'AZURE_SECRET') }}"
    subscription_id: "{{ lookup('env', 'AZURE_SUBSCRIPTION_ID') }}"
    tenant: "{{ lookup('env', 'AZURE_TENANT') }}"
EXPECTED RESULTS

I expect to get the same (or similar output as we get on az account show)

{
  "environmentName": "ExampleCloud",
  "homeTenantId": "12345678-90ab-cdef-1234-567890abcdef",
  "id": "abcdef12-3456-7890-abcd-ef1234567890",
  "isDefault": true,
  "managedByTenants": [
    {
      "tenantId": "abcd1234-ef56-7890-abcd-ef1234567890"
    }
  ],
  "name": "Example Datalake Sandbox",
  "state": "Enabled",
  "tenantId": "12345678-90ab-cdef-1234-567890abcdef",
  "user": {
    "name": "abcd1234-5678-90ab-cdef-1234567890ef",
    "type": "servicePrincipal"
  }
}
ACTUAL RESULTS
The full traceback is:
  File "/tmp/ansible_azure.azcollection.azure_rm_account_info_payload_1l7c52pa/ansible_azure.azcollection.azure_rm_account_info_payload.zip/ansible_collections/azure/azcollection/plugins/modules/azure_rm_account_info.py", line 166, in list_items
  File "/usr/local/lib/python3.9/site-packages/azure/core/paging.py", line 123, in __next__
    return next(self._page_iterator)
  File "/usr/local/lib/python3.9/site-packages/azure/core/paging.py", line 75, in __next__
    self._response = self._get_next(self.continuation_token)
  File "/usr/local/lib/python3.9/site-packages/azure/mgmt/resource/subscriptions/v2019_11_01/operations/_operations.py", line 538, in get_next
    pipeline_response = self._client._pipeline.run(  # pylint: disable=protected-access
  File "/usr/local/lib/python3.9/site-packages/azure/core/pipeline/_base.py", line 213, in run
    return first_node.send(pipeline_request)
  File "/usr/local/lib/python3.9/site-packages/azure/core/pipeline/_base.py", line 70, in send
    response = self.next.send(request)
  File "/usr/local/lib/python3.9/site-packages/azure/core/pipeline/_base.py", line 70, in send
    response = self.next.send(request)
  File "/usr/local/lib/python3.9/site-packages/azure/core/pipeline/_base.py", line 70, in send
    response = self.next.send(request)
  [Previous line repeated 2 more times]
  File "/usr/local/lib/python3.9/site-packages/azure/mgmt/core/policies/_base.py", line 47, in send
    response = self.next.send(request)
  File "/usr/local/lib/python3.9/site-packages/azure/core/pipeline/policies/_redirect.py", line 181, in send
    response = self.next.send(request)
  File "/usr/local/lib/python3.9/site-packages/azure/core/pipeline/policies/_retry.py", line 467, in send
    response = self.next.send(request)
  File "/usr/local/lib/python3.9/site-packages/azure/core/pipeline/policies/_authentication.py", line 113, in send
    self.on_request(request)
  File "/usr/local/lib/python3.9/site-packages/azure/core/pipeline/policies/_authentication.py", line 90, in on_request
    self._token = self._credential.get_token(*self._scopes)
  File "/usr/local/lib/python3.9/site-packages/azure/identity/_internal/get_token_mixin.py", line 83, in get_token
    token = self._acquire_token_silently(*scopes, claims=claims, tenant_id=tenant_id, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/azure/identity/_internal/decorators.py", line 84, in wrapper
    raise auth_error from ex
fatal: [localhost]: FAILED! => {
    "changed": false,
    "invocation": {
        "module_args": {
            "ad_user": null,
            "adfs_authority_url": null,
            "api_profile": "latest",
            "auth_source": "env",
            "cert_validation_mode": null,
            "client_id": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
            "cloud_environment": "AzureCloud",
            "log_mode": null,
            "log_path": null,
            "password": null,
            "profile": null,
            "secret": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
            "subscription_id": "98765432-abcd-1234-cdef-1234567890ab",
            "tenant": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
            "thumbprint": null,
            "x509_certificate_path": null
        }
    },
    "msg": "Failed to list all subscriptions - Authentication failed: __init__() got an unexpected keyword argument 'instance_discovery'"
}
thisguyshouldworkforus commented 6 months ago

@alexander-aps is the business account that should own this. I didn't have a business GitHub (we use GitLab). I'm working with @ericcames.

alexander-aps commented 6 months ago

Yeah, if someone is able to "transfer" this bug report to "https://github.com/alexander-aps" that would be super.

alexander-aps commented 6 months ago

@Fred-sun , this issue is directly related to "https://github.com/ansible-collections/azure/issues/1463" that you're working on with @ericcames -- I am the customer he's referring to. These issues are both surrounding the same module. I am able to authenticate using AzureCLI as called from within a playbook:

---
- name: Proof of concept playbook for Azure
  hosts: localhost
  connection: local
  gather_facts: true
  tasks:
    - name: Authenticate to AzureCloud
      ansible.builtin.command:
        cmd: az login --service-principal --username="{{ lookup('env', 'AZURE_CLIENT_ID') }}" --password="{{ lookup('env', 'AZURE_SECRET') }}" --tenant="{{ lookup('env', 'AZURE_TENANT') }}"
      changed_when: false
      no_log: true

    - name: Set the proper account
      ansible.builtin.command:
        cmd: az account set --subscription="{{ lookup('env', 'AZURE_SUBSCRIPTION_ID') }}"
      changed_when: false

    - name: Show Account Status
      ansible.builtin.command:
        cmd: az account show
      register: azcli_output
      changed_when: false

    - name: Examine output
      ansible.builtin.debug:
        var: azcli_output

    - name: Logout of Azure
      ansible.builtin.command:
        cmd: az logout
      changed_when: false

This works as expected.

Per suggestions in that other thread, I rebuilt my EE to directly install the collection (azure.azcollection) and the Python dependencies in the Containerfile (instead of relying on the requirements.yml file to do it in the ansible-build phase of EE construction). Despite this, I am still not able to use this module.

# Build new EE_BASE_IMAGE that includes updated Python and Terraform
# Start from the Ansible Automation Platform base image
FROM registry.redhat.io/ansible-automation-platform-24/ee-supported-rhel8:latest

# Add useful information
LABEL maintainer "Alexander Snyder <alexander.snyder@aps.com>"

USER root
RUN /usr/sbin/groupadd --system --gid='202' ansible
RUN /usr/sbin/useradd --system --comment='Ansible Execution Environment User' --home-dir='/home/runner' --gid='202' --groups='wheel' --no-create-home --shell='/bin/bash' --uid='202' ansible
#
ENV PATH "/home/runner/.local/bin:/home/runner/.local/lib:$PATH"
RUN /usr/bin/chown -R ansible:root /home/runner
RUN /usr/bin/chmod -R 775 /home/runner
#
# Install Keys & Repositories as required for future steps
RUN rpm --import https://packages.microsoft.com/keys/microsoft.asc
#
COPY ./rpm-files/epel-release-latest-8.noarch.rpm /tmp
RUN /usr/bin/rpm --install /tmp/epel-release-latest-8.noarch.rpm
#
COPY ./repos/microsoft-prod.repo /etc/yum.repos.d/
COPY ./repos/hashicorp.repo /etc/yum.repos.d/
#
# Because Microsoft never makes anything easy
USER ansible
RUN ansible-galaxy collection install azure.azcollection
RUN /usr/bin/pip3 install -r /home/runner/.ansible/collections/ansible_collections/azure/azcollection/requirements-azure.txt --use-feature=2020-resolver
RUN ansible-galaxy collection install azure.azcollection --force
RUN /usr/bin/pip3 uninstall "importlib-metadata>=6.3" --yes
RUN /usr/bin/pip3 install "importlib-metadata<6.3,>=4.6"
#
# Clean up a bit, and install the required packages
USER root
RUN /usr/bin/chown -R root:root /home/runner
RUN /usr/bin/chmod -R 750 /home/runner
RUN /usr/bin/microdnf clean all
RUN /usr/bin/microdnf makecache

Upon using this new EE, and first authenticating with AzureCLI, then trying to use this module to get account info, I have DUPLICATED the error as mentioned in the other thread:

---
- name: Proof of concept playbook for Azure
  hosts: localhost
  connection: local
  gather_facts: true
  tasks:
    - name: Authenticate to AzureCloud
      ansible.builtin.command:
        cmd: az login --service-principal --username="{{ lookup('env', 'AZURE_CLIENT_ID') }}" --password="{{ lookup('env', 'AZURE_SECRET') }}" --tenant="{{ lookup('env', 'AZURE_TENANT') }}"
      changed_when: false
      no_log: true

    - name: Set the proper account
      ansible.builtin.command:
        cmd: az account set --subscription="{{ lookup('env', 'AZURE_SUBSCRIPTION_ID') }}"
      changed_when: false
      no_log: true

    - name: Get facts for current logged in user
      azure.azcollection.azure_rm_account_info:

    - name: Logout of Azure
      ansible.builtin.command:
        cmd: az logout
      changed_when: false
ansible-playbook [core 2.15.9]
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/home/runner/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3.9/site-packages/ansible
  ansible collection location = /runner/requirements_collections:/home/runner/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/bin/ansible-playbook
  python version = 3.9.18 (main, Sep 22 2023, 17:58:34) [GCC 8.5.0 20210514 (Red Hat 8.5.0-20)] (/usr/bin/python3.9)
  jinja version = 3.1.2
  libyaml = True
Using /etc/ansible/ansible.cfg as config file
host_list declined parsing /runner/inventory/hosts as it did not pass its verify_file() method
Parsed /runner/inventory/hosts inventory source with script plugin
Skipping callback 'awx_display', as we already have a stdout callback.
Skipping callback 'default', as we already have a stdout callback.
Skipping callback 'minimal', as we already have a stdout callback.
Skipping callback 'oneline', as we already have a stdout callback.

PLAYBOOK: main.yml *************************************************************
1 plays in main.yml

PLAY [Proof of concept playbook for Azure] *************************************

TASK [Gathering Facts] *********************************************************
task path: /runner/project/main.yml:2
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: root
<127.0.0.1> EXEC /bin/sh -c 'echo ~root && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/runner/.ansible/tmp `"&& mkdir "` echo /home/runner/.ansible/tmp/ansible-tmp-1708990589.8981185-26-267940039825840 `" && echo ansible-tmp-1708990589.8981185-26-267940039825840="` echo /home/runner/.ansible/tmp/ansible-tmp-1708990589.8981185-26-267940039825840 `" ) && sleep 0'
Using module file /usr/lib/python3.9/site-packages/ansible/modules/setup.py
<127.0.0.1> PUT /home/runner/.ansible/tmp/ansible-local-21340m05uj/tmpl13soyzg TO /home/runner/.ansible/tmp/ansible-tmp-1708990589.8981185-26-267940039825840/AnsiballZ_setup.py
<127.0.0.1> EXEC /bin/sh -c 'chmod u+x /home/runner/.ansible/tmp/ansible-tmp-1708990589.8981185-26-267940039825840/ /home/runner/.ansible/tmp/ansible-tmp-1708990589.8981185-26-267940039825840/AnsiballZ_setup.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '/usr/bin/python3.9 /home/runner/.ansible/tmp/ansible-tmp-1708990589.8981185-26-267940039825840/AnsiballZ_setup.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c 'rm -f -r /home/runner/.ansible/tmp/ansible-tmp-1708990589.8981185-26-267940039825840/ > /dev/null 2>&1 && sleep 0'
ok: [localhost]

TASK [Authenticate to AzureCloud] **********************************************
task path: /runner/project/main.yml:7
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: root
<127.0.0.1> EXEC /bin/sh -c 'echo ~root && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/runner/.ansible/tmp `"&& mkdir "` echo /home/runner/.ansible/tmp/ansible-tmp-1708990590.918775-56-60702665459512 `" && echo ansible-tmp-1708990590.918775-56-60702665459512="` echo /home/runner/.ansible/tmp/ansible-tmp-1708990590.918775-56-60702665459512 `" ) && sleep 0'
Using module file /usr/lib/python3.9/site-packages/ansible/modules/command.py
<127.0.0.1> PUT /home/runner/.ansible/tmp/ansible-local-21340m05uj/tmpr6fwu1nk TO /home/runner/.ansible/tmp/ansible-tmp-1708990590.918775-56-60702665459512/AnsiballZ_command.py
<127.0.0.1> EXEC /bin/sh -c 'chmod u+x /home/runner/.ansible/tmp/ansible-tmp-1708990590.918775-56-60702665459512/ /home/runner/.ansible/tmp/ansible-tmp-1708990590.918775-56-60702665459512/AnsiballZ_command.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '/usr/bin/python3.9 /home/runner/.ansible/tmp/ansible-tmp-1708990590.918775-56-60702665459512/AnsiballZ_command.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c 'rm -f -r /home/runner/.ansible/tmp/ansible-tmp-1708990590.918775-56-60702665459512/ > /dev/null 2>&1 && sleep 0'
ok: [localhost] => {
    "censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result",
    "changed": false
}

TASK [Set the proper account] **************************************************
task path: /runner/project/main.yml:13
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: root
<127.0.0.1> EXEC /bin/sh -c 'echo ~root && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/runner/.ansible/tmp `"&& mkdir "` echo /home/runner/.ansible/tmp/ansible-tmp-1708990594.186062-81-209989884495644 `" && echo ansible-tmp-1708990594.186062-81-209989884495644="` echo /home/runner/.ansible/tmp/ansible-tmp-1708990594.186062-81-209989884495644 `" ) && sleep 0'
Using module file /usr/lib/python3.9/site-packages/ansible/modules/command.py
<127.0.0.1> PUT /home/runner/.ansible/tmp/ansible-local-21340m05uj/tmpiy7ncml3 TO /home/runner/.ansible/tmp/ansible-tmp-1708990594.186062-81-209989884495644/AnsiballZ_command.py
<127.0.0.1> EXEC /bin/sh -c 'chmod u+x /home/runner/.ansible/tmp/ansible-tmp-1708990594.186062-81-209989884495644/ /home/runner/.ansible/tmp/ansible-tmp-1708990594.186062-81-209989884495644/AnsiballZ_command.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '/usr/bin/python3.9 /home/runner/.ansible/tmp/ansible-tmp-1708990594.186062-81-209989884495644/AnsiballZ_command.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c 'rm -f -r /home/runner/.ansible/tmp/ansible-tmp-1708990594.186062-81-209989884495644/ > /dev/null 2>&1 && sleep 0'
ok: [localhost] => {
    "censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result",
    "changed": false
}

TASK [Get facts for current logged in user] ************************************
task path: /runner/project/main.yml:28
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: root
<127.0.0.1> EXEC /bin/sh -c 'echo ~root && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/runner/.ansible/tmp `"&& mkdir "` echo /home/runner/.ansible/tmp/ansible-tmp-1708990594.8831482-106-13873823444114 `" && echo ansible-tmp-1708990594.8831482-106-13873823444114="` echo /home/runner/.ansible/tmp/ansible-tmp-1708990594.8831482-106-13873823444114 `" ) && sleep 0'
Using module file /runner/requirements_collections/ansible_collections/azure/azcollection/plugins/modules/azure_rm_account_info.py
<127.0.0.1> PUT /home/runner/.ansible/tmp/ansible-local-21340m05uj/tmpc6g521zt TO /home/runner/.ansible/tmp/ansible-tmp-1708990594.8831482-106-13873823444114/AnsiballZ_azure_rm_account_info.py
<127.0.0.1> EXEC /bin/sh -c 'chmod u+x /home/runner/.ansible/tmp/ansible-tmp-1708990594.8831482-106-13873823444114/ /home/runner/.ansible/tmp/ansible-tmp-1708990594.8831482-106-13873823444114/AnsiballZ_azure_rm_account_info.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '/usr/bin/python3.9 /home/runner/.ansible/tmp/ansible-tmp-1708990594.8831482-106-13873823444114/AnsiballZ_azure_rm_account_info.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c 'rm -f -r /home/runner/.ansible/tmp/ansible-tmp-1708990594.8831482-106-13873823444114/ > /dev/null 2>&1 && sleep 0'
The full traceback is:
Traceback (most recent call last):
  File "/tmp/ansible_azure.azcollection.azure_rm_account_info_payload_3xii3d4v/ansible_azure.azcollection.azure_rm_account_info_payload.zip/ansible_collections/azure/azcollection/plugins/module_utils/azure_rm_common.py", line 256, in <module>
    from azure.storage.blob import BlobServiceClient
ImportError: cannot import name 'BlobServiceClient' from 'azure.storage.blob' (/usr/local/lib/python3.9/site-packages/azure/storage/blob/__init__.py)
fatal: [localhost]: FAILED! => {
    "changed": false,
    "invocation": {
        "module_args": {
            "ad_user": null,
            "adfs_authority_url": null,
            "api_profile": "latest",
            "auth_source": "auto",
            "cert_validation_mode": null,
            "client_id": null,
            "cloud_environment": "AzureCloud",
            "log_mode": null,
            "log_path": null,
            "password": null,
            "profile": null,
            "secret": null,
            "subscription_id": null,
            "tenant": null,
            "thumbprint": null,
            "x509_certificate_path": null
        }
    },
    "msg": "Failed to import the required Python library (ansible[azure] (azure >= 2.0.0)) on 34037756a0ed's Python /usr/bin/python3.9. Please read the module documentation and install it in the appropriate location. If the required library is installed, but Ansible is using the wrong Python interpreter, please consult the documentation on ansible_python_interpreter"
}

PLAY RECAP *********************************************************************
localhost                  : ok=3    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0   

I hope this new information is helpful. You can pull my EE image for your own testing:

podman pull quay.io/z18367a/ee-supported-rhel8_azure
alexander-aps commented 6 months ago

I rebuilt my execution environment to make it as "vanilla" as possible, to help eliminate the issues.

# Build new EE_BASE_IMAGE that includes updated Python and Terraform
# Start from the Ansible Automation Platform base image
FROM registry.redhat.io/ansible-automation-platform-24/ee-supported-rhel8:latest

# Add useful information
LABEL maintainer "Alexander Snyder <alexander.snyder@aps.com>"

# Install Keys & Repositories as required for future steps
RUN rpm --import https://packages.microsoft.com/keys/microsoft.asc
COPY ./repos/microsoft-prod.repo /etc/yum.repos.d/

I built a new image and all was fine. I assembled it with ansible-build:

build_arg_defaults: EE_BASE_IMAGE: 'localhost/ee-supported-rhel8_azure:latest'

dependencies: galaxy: requirements.yml python: requirements.txt system: bindep.txt

ansible_config: 'ansible.cfg'

  - requirements.yml
```yaml
---
collections:
  - name: azure.azcollection
  - name: community.general #Includes Terraform

I pushed this to automation hub, and ran my playbook:

---
- name: Proof of concept playbook for Azure
  hosts: localhost
  connection: local
  gather_facts: true
  tasks:
      # Authenticate to AzureCloud
    - name: Authenticate to AzureCloud
      ansible.builtin.command:
        cmd: az login --service-principal --username="{{ lookup('env', 'AZURE_CLIENT_ID') }}" --password="{{ lookup('env', 'AZURE_SECRET') }}" --tenant="{{ lookup('env', 'AZURE_TENANT') }}"
      changed_when: false

      # Set the proper account
    - name: Set the proper account
      ansible.builtin.command:
        cmd: az account set --subscription="{{ lookup('env', 'AZURE_SUBSCRIPTION_ID') }}"
      changed_when: false

      # Define a virtual machine
    - name: Define an Azure VM
      azure.azcollection.azure_rm_virtualmachine:
        accept_terms: true
        admin_username: "{{ lookup('env', 'AZURE_VM_ADMIN_USERNAME') }}"
        admin_password: "{{ lookup('env', 'AZURE_VM_ADMIN_PASSWORD') }}"
        append_tags: false
        auth_source: env
        boot_diagnostics:
          enabled: false
        created_nsg: false
        force: false
        generalized: false
        image:
          publisher: redhat-limited
          offer: rh-rhel
          sku: rh-rhel9 # rh-rhel8, rh-rhel7
          version: latest
        license_type: 'RHEL_BYOS'
        linux_config:
          disable_password_authentication: false
        name: 'AzurePOC'
        os_disk_caching: 'ReadWrite'
        os_disk_name: 'AzurePOCDisk'
        os_disk_size_gb: 100
        os_type: 'Linux'
        priority: 'None'
        public_ip_allocation_method: 'Disabled'
        resource_group: "{{ lookup('env', 'AZURE_VM_RESOURCE_GROUP') }}"
        security_profile:
          encryption_at_host: false
          security_type: 'TrustedLaunch'
          uefi_settings:
            secure_boot_enabled: true
            v_tpm_enabled: true
        short_hostname: azurepoc
        ssh_password_enabled: true
        ssh_public_keys:
          - path: "/home/{{ lookup('env', 'AZURE_VM_ADMIN_USERNAME') }}/.ssh/authorized_keys"
            key_data: "ssh-ed25519 AAC3NzaE5AAAAIObco4wztB95eYtdO8dLobLRhAGHJaF"
        started: true
        state: present
        subnet_name: "{{ lookup('env', 'AZURE_VM_SUBNET') }}"
        virtual_network_name: "{{ lookup('env', 'AZURE_VM_VIRTUAL_NETWORK') }}"
        vm_size: 'Standard_D4'

      # Logout of Azure
    - name: Logout of Azure
      ansible.builtin.command:
        cmd: az logout
      changed_when: false

The full traceback is:

Traceback (most recent call last):
  File "/tmp/ansible_azure.azcollection.azure_rm_virtualmachine_payload_9p0yvok7/ansible_azure.azcollection.azure_rm_virtualmachine_payload.zip/ansible_collections/azure/azcollection/plugins/module_utils/azure_rm_common.py", line 256, in <module>
    from azure.storage.blob import BlobServiceClient
ImportError: cannot import name 'BlobServiceClient' from 'azure.storage.blob' (/usr/local/lib/python3.9/site-packages/azure/storage/blob/__init__.py)
fatal: [localhost]: FAILED! => {
    "changed": false,
    "invocation": {
        "module_args": {
            "accept_terms": true,
            "ad_user": null,
            "additional_capabilities": null,
            "adfs_authority_url": null,
            "admin_password": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
            "admin_username": "aps_test",
            "allocated": true,
            "api_profile": "latest",
            "append_tags": false,
            "auth_source": "env",
            "availability_set": null,
            "boot_diagnostics": {
                "enabled": false,
                "resource_group": null,
                "storage_account": null,
                "type": null
            },
            "cert_validation_mode": null,
            "client_id": null,
            "cloud_environment": "AzureCloud",
            "created_nsg": false,
            "custom_data": null,
            "data_disks": null,
            "ephemeral_os_disk": null,
            "eviction_policy": null,
            "force": false,
            "generalized": false,
            "image": {
                "offer": "rh-rhel",
                "publisher": "redhat-limited",
                "sku": "rh-rhel9",
                "version": "latest"
            },
            "license_type": "RHEL_BYOS",
            "linux_config": {
                "disable_password_authentication": false
            },
            "location": null,
            "log_mode": null,
            "log_path": null,
            "managed_disk_type": null,
            "max_price": -1.0,
            "name": "AzurePOC",
            "network_interface_names": null,
            "open_ports": null,
            "os_disk_caching": "ReadWrite",
            "os_disk_encryption_set": null,
            "os_disk_name": "AzurePOCDisk",
            "os_disk_size_gb": 100,
            "os_type": "Linux",
            "password": null,
            "plan": null,
            "priority": "None",
            "profile": null,
            "proximity_placement_group": null,
            "public_ip_allocation_method": "Disabled",
            "remove_on_absent": [
                "all"
            ],
            "resource_group": "np-rg-002",
            "restarted": false,
            "secret": null,
            "security_profile": {
                "encryption_at_host": false,
                "security_type": "TrustedLaunch",
                "uefi_settings": {
                    "secure_boot_enabled": true,
                    "v_tpm_enabled": true
                }
            },
            "short_hostname": "azurepoc",
            "ssh_password_enabled": true,
            "ssh_public_keys": [
                {
                    "key_data": "ssh-ed25519 AAC3NzaE5AAAAIObco4wztB95eYtdO8dLobLRhAGHJaF",
                    "path": "/home/aps_test/.ssh/authorized_keys"
                }
            ],
            "started": true,
            "state": "present",
            "storage_blob_name": null,
            "storage_container_name": "vhds",
            "subnet_name": "np-sub-10.10.89.0-26",
            "subscription_id": null,
            "tags": null,
            "tenant": null,
            "thumbprint": null,
            "virtual_network_name": "np-vnet-001",
            "virtual_network_resource_group": null,
            "vm_identity": null,
            "vm_size": "Standard_D4",
            "windows_config": null,
            "winrm": null,
            "x509_certificate_path": null,
            "zones": null
        }
    },
    "msg": "Failed to import the required Python library (ansible[azure] (azure >= 2.0.0)) on 1c7c2195005c's Python /usr/bin/python3.9. Please read the module documentation and install it in the appropriate location. If the required library is installed, but Ansible is using the wrong Python interpreter, please consult the documentation on ansible_python_interpreter"
}

I hope this new information is helpful. You can pull my EE image for your own testing:

podman pull quay.io/z18367a/ee-supported-rhel8_azure
Fred-sun commented 5 months ago

@alexander-aps @thisguyshouldworkforus azure_rm_account_info.py is used to access login account information, so when using this module, you need to have az login, and then use the cli credentials to get login account information through the 'auth_source' setting. After migrating to msgraph earlier, you need to make a few minor changes, see #1510.。 Thank you!


    - name: Get account facts
      azure.azcollection.azure_rm_account_info:
        auth_source: cli
      register: output
    - debug:
        var: output

Return:
"output": {
        "account_info": {
            "environmentName": "AzureCloud",
            "homeTenantId": "xxxxxxxxxx",
            "id": "xxxxxxxxxxxxxxxxxxxxxxxxx",
            "managedByTenants": [],
            "name": "Visual Studio Enterprise",
            "state": "Enabled",
            "tenantId": "8xxxxxxxxxxxxxxxxxxxxxxx",
            "user": {
                "name": "xxxxxxxxxxxxxxxxxx",
                "type": "Member"
            }
        },