Install ansible roles and collections at the same time from the same requirements.yml file?

[jrobison-sb]

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request. Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request. If you are interested in working on this issue or have submitted a pull request, please leave a comment.

Description

It should be possible to install roles and collections together at the same time if you have both roles and collections in a requirements.yml file.

Use Case(s)

It should be possible to install roles and collections together at the same time if you have both roles and collections in a requirements.yml file.

This comment describes the use case in more depth: https://github.com/hashicorp/packer-plugin-ansible/issues/32#issuecomment-1087700918

Potential configuration

If I do it like this:

  provisioner "ansible-local" {
    staging_directory       = "/home/ubuntu/.ansible"
    clean_staging_directory = true
    extra_arguments = [
      "--become",
      "--vault-password-file /etc/ansible/vault_secret_file",
      "--extra-vars '${jsonencode({ in_ci = true })}'",
    ]
    galaxy_file   = "ansible/requirements.yml"
    playbook_dir  = "ansible/"
    playbook_file = "ansible/my_playbook.yml"
  }
}

Then the resulting ansible-galaxy command output will look like this:

    amazon-ebs.my_playbook: Executing Ansible Galaxy: cd /home/ubuntu/.ansible && ansible-galaxy install -r /home/ubuntu/.ansible/requirements.yml -p /home/ubuntu/.ansible/roles
==> amazon-ebs.my_playbook: [WARNING]: The requirements file '/home/ubuntu/.ansible/requirements.yml'
==> amazon-ebs.my_playbook: contains collections which will be ignored. To install these collections run
==> amazon-ebs.my_playbook: 'ansible-galaxy collection install -r' or to install both at the same time run
==> amazon-ebs.my_playbook: 'ansible-galaxy install -r' without a custom install path.
    amazon-ebs.my_playbook: Starting galaxy role install process

It should be possible to do what is described in that output and use 'ansible-galaxy install -r' without a custom install path.

Currently I have to work around it like this:

  # The ansible-local provisioner below doesn't install Ansible collections,
  # it only installs Ansible roles. So we need to upload the requirements.yml
  # file and install collections manually.
  # https://github.com/hashicorp/packer-plugin-ansible/issues/32#issuecomment-1087700918
  provisioner "file" {
    source = "ansible/requirements.yml"
    destination = "/tmp/requirements.yml"
  }

  # The ansible-local provisioner below doesn't install Ansible collections,
  # it only installs Ansible roles. So we need to upload the requirements.yml
  # file and install collections manually.
  # https://github.com/hashicorp/packer-plugin-ansible/issues/32#issuecomment-1087700918
  provisioner "shell" {
    remote_folder  = "/home/ubuntu"
    inline_shebang = "/bin/bash -e"
    inline = [
      "ansible-galaxy install -r /tmp/requirements.yml"
    ]
  }

  provisioner "ansible-local" {
    staging_directory       = "/home/ubuntu/.ansible"
    clean_staging_directory = true
    extra_arguments = [
      "--become",
      "--vault-password-file /etc/ansible/vault_secret_file",
      "--extra-vars '${jsonencode({ in_ci = true })}'",
    ]
    galaxy_file   = "ansible/requirements.yml"
    playbook_dir  = "ansible/"
    playbook_file = "ansible/my_playbook.yml"
  }
}

Potential References

[lbajolet-hashicorp]

Hi @jrobison-sb,

From the looks of it in the comment you linked, the issue got closed with a fix back in 2022, are you using a version of Packer/Ansible plugin that is older than this? Judging by the releases, it was included in v1.0.3 of the plugin, could you make sure you're running this version or above? What's the version of Packer you're using also? If you're using a version before 1.10.0, there's a change you're using an embedded plugin from that version of Packer, so the version of the plugin could be correlated with Packer itself in this case.

[jrobison-sb]

Hi @lbajolet-hashicorp, thanks for your reply.

The affected automation which prompted this issue uses hashicorp/packer:1.7.9.

Is it your understanding that the latest version of ansible-local can already install roles and collections at the same time from the same requirements.yml?

[lbajolet-hashicorp]

Hi @jrobison-sb,

Pretty sure 1.7.9 was embedding a plugin version before 1.0.3 given the release dates, I would suggest you update and try again.

To be clear, I cannot ensure this will fix your problem, and also disclaimer: I'm not super comfortable with Ansible, I don't know the difference between roles and collections, but judging from the linked threads and the commit history, it looks like this got fixed with packer-plugin-ansible 1.0.3 (c.f. commit c64d013146eb224bd).

I would advise you don't continue relying on embedded plugins as they've been removed from more recent versions of Packer (a first part with 1.9.0, and the rest with 1.10.0), we are updating our docs regarding installing plugins to make them more comprehensible, but if you are unfamiliar with how Packer manages plugins, I suggest you read the documentation on that subject.

Hope that helps