Ansible builder fails after upgrade to 3.1.0

[jorgenspange]

Ansible-builder fails after upgrade to 3.1.0 from 3.0.1 due to:

ERROR: Double requirement given: netaddr>=0.10.1 (from -r /tmp/src/requirements.txt (line 13)) (already in netaddr (from -r /tmp/src/requirements.txt (line 4)), name='netaddr')

In my requirements file this is the only thing specified:

ansible-pylibssh>=1.1.0

Tried specifying newer version of netaddr aswell, with no dice.

Once I downgrade to ansible-builder to 3.0.1 everything works.

[sean-m-sullivan]

Running into same issue with netaddr from multiple repos, even if I specify a newer version of netaddr. most Cisco/netcommon collections started this requirement this last week setting more than .10

[Shrews]

Not enough info here for me to reproduce this error. Please provide an example EE definition file, and any associated extras, like collections being used, that reproduces this error.

[sean-m-sullivan]

---
version: 3

dependencies:
  galaxy:
    collections:
      - name: ansible.utils
        version: 5.0.0
      - ansible.network
      - name: ansible.netcommon
        version: 7.0.0
      - name: cisco.ios
        version: 9.0.0
      - name: cisco.iosxr
        version: 10.0.0
      - name: cisco.nxos
        version: 9.0.0
      - name: cisco.asa
        version: 6.0.0
      - name: ansible.posix
      - name: community.general
  python:
    - dnspython
    - netaddr
    - scp # Used for netcommon.net_put
  system:
    - jq
    # - curl Needed but built into UBI:9

images:
  base_image:
    name: >-
      registry.redhat.io/ansible-automation-platform-24/ee-minimal-rhel8:latest

[jorgenspange]

Here is my ee definition file aswell:

---
version: 3

dependencies:
  galaxy:
    collections:
      - name: ansible.posix
      - name: ansible.utils
      - name: ansible.netcommon
      - name: community.general
      - name: cisco.ios
      - name: cisco.iosxr
      - name: cisco.ise
      - name: cisco.nxos
  python:
    - ansible-pylibssh>=1.1.0
    - ciscoisesdk

images:
  base_image:
    name: >-
      registry.redhat.io/ansible-automation-platform-24/ee-minimal-rhel8:latest

[Shrews]

This error is occurring because beginning with version 3.1, we no longer do Python requirements sanitization and simply pass any collection Python requirements on to pip. With 3.0 and sanitization, the Python requirements for netaddr would have been combined and reduced to this entry in the combined requirements.txt file:

netaddr>=0.10.1  # from collection ansible.netcommon,ansible.utils,user

With version 3.1 and no sanitization, we now get this in the combined requirements.txt file:

netaddr  # from collection ansible.netcommon
netaddr>=0.10.1  # from collection ansible.utils
netaddr  # from collection user

Older versions of pip cannot handle these duplicate entries and produce the Double requirement error you see. The version of pip in ee-minimal-rhel8:latest is 20.2.4 and is too old to handle this situation.

Since ansible-builder is now out of the business of trying to do anything with these requirements, I can think of three ways of dealing with this moving forward:

1. Use a base image with a newer pip (not quite certain which version of pip fixes this issue)
2. Upgrade pip with the additional_build_steps EE directive
3. Use the new exclude option found under dependencies (see here) in 3.1 to ignore netaddr and just add it to your user requirements.txt

I think of these, solution number 2 is best. Here's what I added to the EE file to make it work for me:

additional_build_steps:
  prepend_base:
    - RUN pip3 install -U pip

[Shrews]

Closing per above.

[sean-m-sullivan]

I think the problem is that those of us using upstream ansible-builder, with downstream images have the version of pip that it ships with, that is breaking it.

In addition This is a breaking change, that is also not mentioned in the changelog

[sean-m-sullivan]

Or also, could fix the conflict by making them match.

[Shrews]

I think the problem is that those of us using upstream ansible-builder, with downstream images have the version of pip that it ships with, that is breaking it.

Yes, that's what I said in my previous comment. The version of pip in the base image is too old.

In addition This is a breaking change, that is also not mentioned in the changelog

It is mentioned in the details on PR #664, but yes, that info probably should have been extracted from the PR and exposed more in the release notes. I'll see about editing those notes.