Azure image gallery version can't set exclude_from_latest to false

[Akasurde]

From @jghal on Jun 19, 2020 22:11

SUMMARY

Setting publishing_profile.exclude_from_latest: false in azure_rm_galleryimageversion task produces bad API call resulting in task failure.

ISSUE TYPE

• Bug Report

COMPONENT NAME

azure_rm_galleryimageversion

ANSIBLE VERSION

ansible 2.9.9
  config file = /work/ict-provisioner/ansible.cfg
  configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3.8/site-packages/ansible
  executable location = /usr/bin/ansible
  python version = 3.8.2 (default, Feb 29 2020, 17:03:31) [GCC 9.2.0]

CONFIGURATION

DEFAULT_CALLBACK_WHITELIST(/work/ict-provisioner/ansible.cfg) = ['time', 'profile_tasks']
HOST_KEY_CHECKING(/work/ict-provisioner/ansible.cfg) = False
INTERPRETER_PYTHON(/work/ict-provisioner/ansible.cfg) = /usr/bin/python

OS / ENVIRONMENT

Play hosts is localhost, which is an Alpine 3.11 derivative container

STEPS TO REPRODUCE

- name: Promote SIG version
  hosts: localhost
  connection: local
  tasks:
        - name: Create or update gallery image Version
          azure_rm_galleryimageversion:
            resource_group: "{{ rg_name }}"
            gallery_name: "{{ img_gal_name }}"
            gallery_image_name: "{{ image_name }}"
            name: "{{ release }}"
            location: "{{ location }}"
            publishing_profile:
              end_of_life_date: "{{ '%Y-%m-%d %H:%M:%S' | strftime(ansible_date_time.epoch|int + 60*86400) }}"
              exclude_from_latest: no
              replica_count: 1
              storage_account_type: "{{ builder.managed_image_storage_account_type }}"

EXPECTED RESULTS

Image version is created in Gallery and included in latest.

ACTUAL RESULTS

"msg": "Error creating the GalleryImageVersion instance: Azure Error: BadRequest\nMessage: Could not find member 'exclude_from_latest' on object of type 'GalleryImageVersionPublishingProfile'. Path 'properties.publi
shingProfile.exclude_from_latest', line 1, position 86.\nTarget: galleryImageVersion.properties.publishingProfile.exclude_from_latest"

Running with a high degree of verbosity, it seems that when exclude_from_latest is set to true the generated API call JSON includes publishing_profile.excludeFromLatest but when setting it to false the API call JSON is sending publishing_profile.exclude_from_latest.

Copied from original issue: ansible/ansible#70187

[Akasurde]

cc @jghal

[jghal]

So I had meant to open a second copy of the ticket here, so thanks for cleaning up my mess of duplicates!

Do fixes made here in the collection get back-ported into Ansible 2.9?

[Fred-sun]

@Akasurde I'm not sure what caused the problem, but was it successful when I configured target_Regions and Managed_image? Please try again. Thank you very much!

- name: Create a gallery Image Version.
  azure_rm_galleryimageversion:
    resource_group: "{{ resource_group }}"
    gallery_name: myGallery{{ rpfx }}
    gallery_image_name: myImage
    name: 10.1.3
    location: eastus
    publishing_profile:
      end_of_life_date: "{{ '%Y-%m-%d %H:%M:%S' | strftime(ansible_date_time.epoch|int + 60*86400) }}"
      exclude_from_latest: yes
      replica_count: 3
      storage_account_type: Standard_LRS
      target_regions:
        - name: eastus
          regional_replica_count: 1
      managed_image:
        name: "{{ image_name }}"
        resource_group: "{{ resource_group }}"

[jghal]

the bug is when you set exclude_from_latest: no (or false instead of no).

[Fred-sun]

the bug is when you set exclude_from_latest: no (or false instead of no).

Hi jghal I tried it locally, no matter whether the value of end_of_life_date is False or Ture, All test pass! Thank you very much!

[jghal]

Have you tested with the version of the module in the Ansible stable-2.9 release branch? I feel like this code is in a weird nexus state where Ansible 2.9 appears to ship with the code from github.com/ansible/ansible but all issues reported against it are closed because development is moved to here. So I don't know if I can expect that if it's fixed here that it will make it to a 2.9.x release or not.

[Fred-sun]

Have you tested with the version of the module in the Ansible stable-2.9 release branch? I feel like this code is in a weird nexus state where Ansible 2.9 appears to ship with the code from github.com/ansible/ansible but all issues reported against it are closed because development is moved to here. So I don't know if I can expect that if it's fixed here that it will make it to a 2.9.x release or not.

I tried it in 2.9, thank you very much!

[jghal]

Maybe it is dependent on version of Azure python modules installed then? I setting up my environment with this PIP requirements file

ansible[azure]>=2.9,<2.10
ansible-lint==4.2.0
flake8==3.8.1
hvac>=0.10,<0.11
molecule==3.0.4
molecule-azure==0.3
molecule-ec2==0.2
pytest==5.4.2
pytest-molecule==1.2.5
pypsrp>=0.4,<0.5
pywinrm>=0.4,<0.5
testinfra==5.1.0

I do not have the full azure-cli installed as we haven't found (and according to https://github.com/ansible/ansible/issues/60153 there likely will never be) a combination of ansible[azure] and azure-cli PIP modules that can truly live together.

[jghal]

So I just installed the collection to see if it works that way, and set my playbook to use it, and see errors like this

"Failed to import the required Python library (ansible[azure] (azure >= 2.0.0)) An exception occurred during task execution. To see the full traceback, use -vvv. The error was: ModuleNotFoundError: No module named 'azure.mgmt.privatedns'

But I have installed ansible[azure] PIP module, and that doesn't seem to include a module called azure-mgmt-privatedns. So exactly what version of that am I supposed to install?

I've pasted my full playbook, a listing of all installed PIP modules, and the errors I see using the collection versus with what's shipped with Ansible 2.9.x

https://gist.github.com/jghal/67eeb8d9a183961508df24b713fa38a9

[Fred-sun]

@jghal Please install azure-mgmt-privatedns by "sudo pip3 install azure-mgmt-privatedns==0.1.0" or "sudo pip3 install -r ./ansible_collections_azure/requirements-azure.txt". Thank you very much!

[jghal]

Installing Ansible >=2.9,<2.10 with PIP virtualenv along with the requirements file in this GitHub repo then using the collection in my playbook is working. I take it you also expect this combination to work for the version of the Azure modules shipped with Ansible 2.9 as well? Is there a PIP module we can install instead of ansible[azure] to get these working dependencies instead of fetching the requirements file from this repo?

[Fred-sun]

@jghal Sorry for forgetting to reply to your question. Because ansible is a powerful tool, it needs more dependencies. The requirements-azure.txt can let users know more clearly which dependencies need to be installed to ensure the normal operation of ansible. I think this is necessary. Thank you！ It can be installed like below：

    1  sudo apt update
    2  apt install python3-pip
    3  sudo pip3 install ansible
    4  ansible-galaxy collection install azure.azcollection
    5  sudo pip3 isntall -r /root/.ansible/collections/ansible_collections/azure/azcollection/requirements-azure.txt

[jghal]

Ah, it hadn't occurred to me that it would be included in the distribution that ansible-galaxy fetchs. Thanks.

[el-memer]

Hi @Fred-sun, are you working on this issue ? I would like to contribute (and begin with this one :) ) But i don't want to interfere in something you're working on !

[jghal]

@el-memer I'm not seeing this bug in Ansible 2.9 when I use this Collection with the PIP requirements file from this Collection. I'm assuming that it's not going to be fixed for the Ansible 2.9 bundled module code or the ansible[azure] PIP module. It seems that Ansible at large deprecated and stopped supporting all the modules bundled in 2.9 before the replacement Collections and Ansible 2.10 were officially released (evidenced by all the bugs in the ansible/ansible GH project that were closed and referred to be reopened in the relevant Collection's new GH repo).

[Fred-sun]

@Akasurde This issue should have been fixed and will return the correct result whether "exclude_from_latest" is set to yes or no. You can refer to YAML below. Thanks!

---
- name: Using Azure collection
  hosts: localhost
  collections:
    - azure.azcollection
  tasks:
    - name: Create a simple gallery Image Version with  I(exclude_from_latest=yes)
      azure_rm_galleryimageversion:
        resource_group: "{{ resource_group }}"
        gallery_name: myGallery{{ rpfx }}
        gallery_image_name: myImage
        name: 10.1.3
        location: eastus
        publishing_profile:
          end_of_life_date: "2050-10-01t00:00:00+00:00"
          exclude_from_latest: yes ----------- You can update to I(exclude_from_latest=no)
          replica_count: 3
          storage_account_type: Standard_LRS
          target_regions:
            - name: eastus
              regional_replica_count: 1
            - name: westus
              regional_replica_count: 2
              storage_account_type: Standard_ZRS
          managed_image:
            name: testimagea
            resource_group: "{{ resource_group }}"

[wfe8006]

Unfortunately it still doesn't work in Ansible 2.9:

      azure_rm_galleryimageversion:
        resource_group: "{{ gallery_resource_group }}"
        gallery_name: "{{ gallery_name }}"
        gallery_image_name: "{{ default_gallery_image_name }}"
        name: "{{ new_gallery_image_version }}"
        publishing_profile:
          end_of_life_date: "2030-12-31t00:00:00+00:00"
          exclude_from_latest: no
          replica_count: 1
          storage_account_type: Standard_LRS
          target_regions:
            - name: Australia Southeast
              regional_replica_count: 1
          managed_image:
            name: "{{ latest_image_name }}"
            resource_group: "{{ default_vm_resource_group }}"

fatal: [localhost]: FAILED! => {"changed": false, "msg": "Error creating the GalleryImageVersion instance: Azure Error: BadRequest\nMessage: Could not find member 'exclude_from_latest' on object of type 'GalleryImageVersionPublishingProfile'. Path 'properties.publishingProfile.exclude_from_latest', line 1, position 536.\nTarget: galleryImageVersion.properties.publishingProfile.exclude_from_latest"}

_______________ 
<  Tower 3.8.6  >
 --------------- 
        \   ^__^
         \  (oo)\_______
            (__)      A )\/\
                ||----w |
                ||     ||

Ansible 2.9.15
Copyright © 2023 Red Hat, Inc.
Visit [Ansible.com](http://www.ansible.com/) for more information.

@Fred-sun

[Fred-sun]

@wfe8006 ansible version is higher than 2.9, plus you can install the latest ansible-collections, and dependency packages have been updated to the latest version. I suggest you try it. Thanks!

Install:
    ansible-galaxy collection install azure.azcollection --force
    pip3 install -r ~/.ansible/collections/ansible_collections/azure/azcollection/requirements-azure.txt

playbook:
- name: Create virtual network
  azure_rm_virtualnetwork:
    resource_group: "{{ resource_group }}"
    name: "testVnet{{ rpfx }}"
    address_prefixes: "10.0.0.0/16"

- name: Add subnet
  azure_rm_subnet:
    resource_group: "{{ resource_group }}"
    name: "testSubnet{{ rpfx }}"
    address_prefix: "10.0.1.0/24"
    virtual_network: "testVnet{{ rpfx }}"

- name: Create public IP address
  azure_rm_publicipaddress:
    resource_group: "{{ resource_group }}"
    allocation_method: Static
    name: "testPublicIP{{ rpfx }}"

- name: Create virtual network inteface cards for VM A and B
  azure_rm_networkinterface:
    resource_group: "{{ resource_group }}"
    name: "vmforimage{{ rpfx }}nic"
    virtual_network: "testVnet{{ rpfx }}"
    subnet: "testSubnet{{ rpfx }}"

- name: Create VM
  azure_rm_virtualmachine:
    resource_group: "{{ resource_group }}"
    name: "vmforimage{{ rpfx }}"
    location: eastus
    admin_username: testuser
    ssh_password_enabled: false
    ssh_public_keys:
      - path: /home/testuser/.ssh/authorized_keys
        key_data: "ssh-rsa************"
    vm_size: Standard_B1ms
    network_interfaces: "vmforimage{{ rpfx }}nic"
    image:
      offer: UbuntuServer
      publisher: Canonical
      sku: 16.04-LTS

- name: Get VM facts
  azure_rm_virtualmachine_info:
    resource_group: "{{ resource_group }}"
    name: "vmforimage{{ rpfx }}"
  register: output

- name: Generalize VM
  azure_rm_virtualmachine:
    resource_group: "{{ resource_group }}"
    name: "vmforimage{{ rpfx }}"
    generalized: yes

- name: Create custom image
  azure_rm_image:
    resource_group: "{{ resource_group }}"
    name: testimagea
    source: "vmforimage{{ rpfx }}"

- name: Create or update a simple gallery
  azure_rm_gallery:
    resource_group: "{{ resource_group }}"
    name: myGallery{{ rpfx }}
    location: eastus
    description: This is the gallery description.

- name: Create a gallery image
  azure_rm_galleryimage:
    resource_group: "{{ resource_group }}"
    gallery_name: myGallery{{ rpfx }}
    name: myImage
    location: eastus
    os_type: linux
    os_state: generalized
    identifier:
      publisher: myPublisherName
      offer: myOfferName
      sku: mySkuName
    description: Image Description

- name: Create or update a simple gallery Image Version.
  azure_rm_galleryimageversion:
    resource_group: "{{ resource_group }}"
    gallery_name: myGallery{{ rpfx }}
    gallery_image_name: myImage
    tags:
      key3: value3
    name: 10.1.3
    location: eastus
    publishing_profile:
      end_of_life_date: "2050-10-01t00:00:00+00:00"
      exclude_from_latest: no
      replica_count: 1
      storage_account_type: Standard_LRS
      target_regions:
        - name: eastus
          regional_replica_count: 1
      managed_image:
        name: testimagea
        resource_group: "{{ resource_group }}"

- name: Get a simple gallery Image Version info.
  azure_rm_galleryimageversion_info:
    resource_group: "{{ resource_group }}"
    gallery_name: myGallery{{ rpfx }}
    gallery_image_name: myImage
    name: 10.1.3

[Fred-sun]

@wfe8006 My playbook in the comments, I did not encounter your error. Could you please provide the detailed log? Thank you very much!