ansible-builder 3.1.0 encountering "ERROR: Double requirement given: dnspython"

ISSUE TYPE

Bug Report

SUMMARY

Encountering Issue with ansible-builder building an execution environment:

Ignoring ipaddress: markers 'python_version < "3.3"' don't match your environment
ERROR: Double requirement given: dnspython (from -r /tmp/src/requirements.txt (line 74)) (already in dnspython>=2.0.0 (from -r /tmp/src/requirements.txt (line 63)), name='dnspython')
Error: building at STEP "RUN /output/scripts/assemble": while running runtime: exit status 1

Issue occurs whether dnspython is in the dependencies python list or not.
Environment is:

Fedora 40
ansible-builder 3.1.0
python 3.12
ansible-navigator 24.2.0 (this specific version to get around #1766

ANSIBLE-NAVIGATOR VERSION

ansible-navigator 24.2.0

CONFIGURATION

---
ansible-navigator:
  execution-environment:
    image: <our custom EE>
    pull:
      policy: missing
  playbook-artifact:
    enable: false
  logging:
    file: ~/ansible-navigator.log

LOG FILE

Ignoring ipaddress: markers 'python_version < "3.3"' don't match your environment
ERROR: Double requirement given: dnspython (from -r /tmp/src/requirements.txt (line 74)) (already in dnspython>=2.0.0 (from -r /tmp/src/requirements.txt (line 63)), name='dnspython')
Error: building at STEP "RUN /output/scripts/assemble": while running runtime: exit status 1

STEPS TO REPRODUCE

Build Fedora 40 system

Run

python3 -m pip install ansible-navigator==24.2.0 --user

Define your execution environment as follows:
```
---
version: 3
```

build_arg_defaults: ANSIBLE_GALAXY_CLI_COLLECTION_OPTS: '--upgrade'

dependencies: galaxy: collections:

ansible.controller
ansible.posix
ansible.utils
ansible.windows
azure.azcollection
community.windows
microsoft.ad
redhat.rhel_system_roles
redhat.satellite
vmware.vmware_rest
community.crypto
cloud.vmware_ops python:
- selinux
- dnspython
- psutil
- netaddr
- pyjwt system:
- findutils [platform:rpm]

images: base_image: name: registry.redhat.io/ansible-automation-platform-24/ee-minimal-rhel8:latest

additional_build_files:

src: /home/bob/.ansible.cfg dest: configs

additional_build_steps: prepend_galaxy:

ADD _build/configs/.ansible.cfg ${HOME}/.ansible.cfg

options: package_manager_path: /usr/bin/microdnf

4. Run the following to build the Execution Environment:

ansible-builder build --verbosity ${EE_VERBOSITY} --prune-images --tag ${EE_NAME}:${EE_VERSION} --tag ${EE_NAME}:latest


##### EXPECTED RESULTS

<!-- What did you expect to happen when running the steps above? -->
It is expected that ansible-builder will be able to resolve the python dependencies for dnspython and build an Execution Environment.  

##### ACTUAL RESULTS

<!-- What actually happened? -->

Ignoring ipaddress: markers 'python_version < "3.3"' don't match your environment ERROR: Double requirement given: dnspython (from -r /tmp/src/requirements.txt (line 74)) (already in dnspython>=2.0.0 (from -r /tmp/src/requirements.txt (line 63)), name='dnspython') Error: building at STEP "RUN /output/scripts/assemble": while running runtime: exit status 1


##### ADDITIONAL INFORMATION

<!-- Include any links screenshots or other
additional information like execution-envirnment details
if enabled. You can use command
"ansible-navigator images <name-of-container-image>"
to introspect execution environment -->

When using ansible-builder 3.0.1, the Execution Environment is built successfully without error.  ansible-builder 3.1.0 seems to be the problem.  

I scanned through the requirements.txt files for all the collections and only found 1 reference to dnspython:  

microsoft.ad v1.7.1 requirements.txt

dnspython >= 2.0.0



There should not be 2 entries for dnspython in conflict as I am specifying dnspython in the execution_environment.yml without a version.  Further, even if I remove the reference to dnspython from the execution_environment.yml, I still get the error.

ansible / ansible-navigator