search

ansible-community / community-topics

[Moved to Ansible Forum] Discussions for Ansible Community Meetings
https://docs.ansible.com/ansible/devel/community/steering/community_steering_committee.html#community-topics-triage
GNU General Public License v3.0
35 stars 9 forks source link

Community execution environments #31

Closed cidrblock closed 5 months ago

cidrblock commented 3 years ago

Summary

Discuss community EE's:

1) prebuilt and published? 2) just sample manifest files? 3) discipline specific?

Additional thoughts:

This does not need to be a voting issue, probably best a conversation to surface thoughts and questions.

Some possible discussion topics:

What would the goal of embracing EEs be within the community:

How far down the EE path?

From comments:

Maybe related:

Should one or more steering committee members sponsor this effort and represent it moving forward?

samccann commented 3 years ago

As we consider this, we should also consider the docs impact. What if anything do we need for community EE docs. Is it just a list of what collections are included and go find the full docs on docs.ansible.com (based on the full Ansible package release) or something else?

cidrblock commented 3 years ago

Revisit closer to fest

tomoliveri commented 3 years ago

I've had some thoughts on this topic, and think its really worth investing some time in. I've put together an 'all in one', which is actually 'as many as I can get working' https://github.com/tomoliveri/ansible_aio_ee

You're spot on @cidrblock not all community collections include their requirements.txt, which makes things tricky.

It would be really brilliant to be able to allow people to add an execution environment image from quay.io/ansible_ee/collectionname And have those images all automatically generated.

Collections have massively increased the barrier to entry for Ansible, we should really invest some effort in mitigating that pain.

felixfontein commented 3 years ago

When we discussed this in the community meeting, the main result was "this really needs better docs". Also for larger collections such as community.general, creating something like requirements.txt / bindep.txt is a huge undertaking that needs better tooling.

tomoliveri commented 3 years ago

I’ll work on automating scanning collections for requirements within plugins/modules (documentation) and automatically produce pull requests for their (commented / formatted) requirements.txt

Doesn’t solve the documentation problem but it’s a step towards it.

On Sun, 8 Aug 2021 at 1:01 am, Felix Fontein @.***> wrote:

When we discussed this in the community meeting, the main result was "this really needs better docs". Also for larger collections such as community.general, creating something like requirements.txt / bindep.txt is a huge undertaking that needs better tooling.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/ansible-community/community-topics/issues/31#issuecomment-894665647, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAXPEU2GC3J2PR3J4C73AG3T3VDGJANCNFSM477HMSVQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email .

felixfontein commented 3 years ago

I’ll work on automating scanning collections for requirements within plugins/modules (documentation) and automatically produce pull requests for their (commented / formatted) requirements.txt

That generally won't work well, since these are free-text fields (also if not, they sometimes use system package names, sometimes use Python package names). Also many modules cannot be run in the EE but must be run on the remote system, and thus their requirements must not be included in requirements.txt.

tadeboro commented 3 years ago

I did some thinking about this topic a while back when I was making sure one of the certified collections I help maintain had all relevant metadata needed when building an execution environment. And the conclusion I came to at that time w that maintaining execution environments is not something collection maintainers should or actually can do.

Collection maintainers have no control over the playbooks and roles users write. And while packing up the collection into an execution environment is relatively simple, such containers are of little use to playbook authors if they need to use content from more than a single collection. This reason, combined with the fact that it is not possible to use more than one execution environment to run a playbook, severely limits the usefulness of prebuilt execution environments.

In my opinion, maintainers should do their best to equip collections with the correct metadata, and leave execution environment building to users. But even that might be a tall order for large collections such as the community.general where there is a lot of content with possibly incompatible control node requirements.

pabelanger commented 3 years ago

IMO, what would need to happen is something like: https://github.com/ansible/network-ee Which is the meta data for execution enviornments, along with a set of collections. Once that is created, testing would need to be setup, build atop of the community ee. Obviously you'd never add all the collections to start, but as each collection that wants to be part of the community EE, you'd likely want to build a gating system to ensure all the required dependencies installed.

The main thing that is missing upstream right now, is a global constraint system. I'm working on a proposal to push that upstream to redhat maintained collection content, which could also be used by community if wanted.

felixfontein commented 2 years ago

Now that AnsibleFest is happening: are the promised better docs available by now?

felixfontein commented 2 years ago

Today I randomly stumbled about some docs which look promising:

These also link to more information on the bindep.txt files: https://docs.opendev.org/opendev/bindep/latest/readme.html

felixfontein commented 2 years ago

There are a couple of community connections having EE support by now (not all of these available in releases yet):

Right now testing EEs with AZP seems to be (next to) impossible since AZP tests are run inside docker containers, which makes using ansible-navigator run impossible. But with GHA testing works fine.

felixfontein commented 2 years ago

For community.general I've added two propositions on how we can make it EE ready here: https://github.com/ansible-collections/community.general/issues/4512 Please comment on these! Obviously it won't be an easy ride (see @tadeboro's comment: [...] where there is a lot of content with possibly incompatible control node requirements), but it might be worth a try.

russoz commented 2 years ago

Herding the dependency cats is insane IMO. I think we should make it as easy as possible for devs to create their own EE containers, with only the specific dependencies they need.

samccann commented 5 months ago

We now have a pair of community-ees so closing out this community topic.

For future discussions/enhancement suggestions, please open a forum topic as the new place to discuss in the Ansible community and thanks!