Request - include python requirement.txt in build

[AlanCoding]

SUMMARY

For the purpose of using this collection in execution environments in the future, we would like to have the python requirements for module usage in the collection build.

I believe that would require adding requirements.txt and its dependent files to what is copied to the build folder:

https://github.com/theforeman/foreman-ansible-modules/blob/8ebda3f5dcafeea277e38c58d903614659f5b776/Makefile#L94-L95

You can put this in a different place if needed, we would just like for it to be in there somewhere.

ISSUE TYPE

• Feature Idea

[evgeni]

Thanks @AlanCoding, so you're finally implementing my suggestion from a year ago :D (https://github.com/ansible/ansible/issues/62733)

Opened a PR that should add the runtime deps to the dist.

[evgeni]

btw, are there plans to make (python style?) optional requirements? e.g. all our modules need apypie, but only some need rpm, debian and ipaddress libraries, and it'd be cool to be able to express that.

[AlanCoding]

I get where you're coming from, it's just too early in development to get into that problem.

Think of it this way - we are building some tooling to add layers to a container image so that a collection works. The end goal is to know "this image works for collections x, y, and z". There's not a strong disincentive to installing more stuff, and needing to switch to a different image depending on what you're running would be painful. We want 1 image to solve the needs of many collections, each of which have many modules that have several types of needs.

The only major reason to not include something is if it poses a risk of conflicts with other software. This is a credible scenario. I want to start installing stuff in multi-collection environments and see what breaks. From there, we will know better how to design a feature to sub-select or exclude certain things.

[evgeni]

My concern was slightly different.

We have a module that uses the Python RPM bindings. Those don't exist "natively" on PyPI.

There is, however, a wrapper package rpm-installer that works in many cases and sometimes just fails to install for no apparent reason. Because of that, I'd be reluctant to add it to the main requirements file, but at the same time it'd be good to have it documented somewhere in machine readable form. Today we just rely on the bindings available on the users systems from the OS.

[AlanCoding]

So I guess that may be the rpm-py-installer package, here are your requirements: entries from modules and files that I got from scraping everyone's data.

  "theforeman.foreman": [
    "apypie>=0.0.5",
    "-r requirements-common.txt",
    "ipaddress; python_version < '3.3'",
    "vcrpy",
    "ansible_runner",
    "python-debian",
    "rpm-py-installer",
    "docker",
    "git+https://github.com/Apipie/apypie.git@master#egg=apypie",
    "-r https://github.com/ansible/ansible/raw/devel/test/lib/ansible_test/_data/requirements/sanity.pylint.txt",
    "-r https://github.com/ansible/ansible/raw/devel/test/lib/ansible_test/_data/requirements/sanity.validate-modules.txt",
    "apypie",
    "whitelisting in configuration",
    "requests (python library)",
    "ipaddress",
    "python-debian (For deb Package upload)",
    "rpm (For rpm upload)"
  ],

...this has to be taken with a grain of salt.

So it feels like requirements.txt are like the minimum requirements, and there are "extra" requirements that someone may want to add. Those may not always be resolvable to simple yum or pip requirements. We have a section in the execution environment spec to give extra build steps, but we don't have anything hooked up to allow a collection to provide some extra steps that the execution environment spec may hook into. Maybe that will make sense at some point.

[evgeni]

Ah, cool. Yeah. Then I think having only the "core" deps in requirements.txt makes sense. Thanks!