search

theforeman / foreman-ansible-modules

Ansible modules for interacting with the Foreman API and various plugin APIs such as Katello
GNU General Public License v3.0
146 stars 163 forks source link

shipping roles/playbooks that use our modules? #535

Closed evgeni closed 4 years ago

evgeni commented 4 years ago
SUMMARY

Last week, @sthirugn brought up the question of where to put roles and playbooks that use our modules. Obviously, every user can drop their stuff wherever they want, but at the same time it would probably be cool to ship a few examples how the individual modules can be used together.

There are already roles out there (https://github.com/sean797/ansible-role-configure-katello/, https://github.com/flyemsafe/satellite-day-two-ops) who use the modules, but I think it'd be cool to ship at least some of them as part of this collection.

I think it would be beneficial for the adoption of the collection to provide a few basic workflows that can be reused, from the top of my head:

Obviously there is no one-size-fits-all and thus we'd need to agree on how much options/flexibility we want to allow before saying "please fork/write your own, this is beyond scope".

Or is this totally crazy talk and this collection should only be about the modules and we should start another for the roles/playbooks?

cc @sean797 and @flyemsafe as the authors of existing roles

ISSUE TYPE
sthirugn commented 4 years ago

+1 for shipping a few roles as part of this collection. There are several use cases which are similar across deployments which can use Roles with variables externalized for easy reuse.

ehelms commented 4 years ago

I am indifferent as to whether there is a separate collection, but I would like to see all of this living in this one repository. There are benefits to centralizing development to this repository for visibility amongst all contributors and users. Can collections require other collections?

I think we should be open to all workflow role proposals, and use our collective knowledge of Foreman to decide what makes sense to be a generic role and what is hitting an edge case that is better off living in a users github repository. Starting with core workflows that address building a server, configuring the server and maintaining said server is where I would like to see the initial focus.

So far I have stuck to thinking of roles. You mentioned playbooks as well. I would need to see some examples of playbooks being built (where roles did not suffice) to better understand why we would want to include those.

ehelms commented 4 years ago

This may or may not be the right place to discuss our naming conventions as contributions are made. The PR (https://github.com/theforeman/foreman-ansible-modules/pull/548) sparks this note as it offers a role with Satellite naming conventions. I tend towards enforcing naming that focus on the community projects rather than any particular downstream products unless something is specific enough to said product without being generalizable (which I think should be nearly non-existent).

sthirugn commented 4 years ago

This may or may not be the right place to discuss our naming conventions as contributions are made. The PR (#548) sparks this note as it offers a role with Satellite naming conventions. I tend towards enforcing naming that focus on the community projects rather than any particular downstream products unless something is specific enough to said product without being generalizable (which I think should be nearly non-existent).

Do you mean any role which will be accepted in this repo should follow only foreman/katello naming convention? Then we need to define a process (create a new repo?) to store Satellite specific roles.

jwbernin commented 4 years ago

I'm more than happy to change the naming convention - the reason behind the convention I used is more laziness than anything, as I was testing my work against a Satellite. I don't believe there's anything specific to Satellite versus Foreman, although there are things specific to RH CDN versus external (non-RH) repositories / products.

ekohl commented 4 years ago

Without thinking too much about it, I'm slightly leaning to not shipping roles here and sticking to primitives. Roles and playbooks are worksflows and thus subject to opinion where primitives aren't. The reference collection with roles can depend on this collection.

evgeni commented 4 years ago

re naming: with this being @theforeman, I think we should stick to Foreman/Katello naming. if we ever find something that is really Satellite specific (I doubt it), it can go under the @redhatsatellite umbrella. but we probably should defer the naming convention to a later point.

sthirugn commented 4 years ago

@ekohl We don't know at this point about how many roles/playbooks we would create in the future. If it is just a handful, I think it is okay that they live here. If there are going to be too many roles in the future, it makes sense to create a separate project and move all implementations like roles, playbooks there. I think at this point, it is worth to start accepting roles/playbooks here.

evgeni commented 4 years ago

@Fobhep @mdellweg @bagasse what do you think?

mdellweg commented 4 years ago

I think, i'm with @ekohl , because once we have a role here, it would be unkind to move it elsewhere. OTOH merging another collection with this one, might be easier. In any case it's hard to judge on this without an actual example.

Fobhep commented 4 years ago

I think the basic idea behind Ansible collections is clearly to deliver modules AND roles together, so that a user can only execute workflows very simply by external parameterization, without having to worry about the underlying modules.

If dependencies of a collection can be installed easily and automatically - which I couldn't find out at the moment due to a lack of documentation, does anyone know if there is anything available? - then nothing speaks against splitting.

mdellweg commented 4 years ago

I just had a brief discussion with @Fobhep . And maybe it's a question of quality. If we add roles here, we kind of promise to keep them in a working state with the corresponding modules. And for some roles that are more atomic (and more opinionated) than others this might be easier. Not generally against it, but we should have a good idea of when to say yes or no thank you.

Fobhep commented 4 years ago

Based on this, it might be an idea to start a "dirty" role pool in another repo and then migrate maintained roles here, as soon as we have an idea of what we want to accept? :thinking:

ehelms commented 4 years ago

My summed up take away is:

a) Collections provide a way to present users with multiple Ansible types to enable their experience b) We want to balance developer overhead and visibility with quality, supported content c) We need to draw a line between supported and unsupported

If I look at our modules we support:

a) tested modules that meet our standards b) modules that target core APIs

Can we apply these to roles? The hardest part might be defining 'target core workflows' but I think we could all agree on a few up front.

a) managing content b) building a host c) configuring and maintaining a host

evgeni commented 4 years ago

Yes, I think the emphasis should be on quality and core.

Given any role using this modules would depend on a specific version of the modules (until we define the module interfaces as stable, which we didn't yet), having those roles in this repo would ease saying "this role works with this module". It will also make easier to test if changes to modules break the roles.

But to answer the question raised by @Fobhep yes, collections can depend on each other, and the Ansible tooling will install dependencies automatically for you.

And yes, a staging/incubator area might be interesting.

I (personally) would like to provide opinionated roles/workflows for the following usecases:

All of these should aim at simple, lab-like environments (so no crazy CCVs with 100 CVs in them, or 50 subnets with 3 domains each). I think that would suffice for 80% of users. The other 20% can fork/use the roles as examples. (having written that, I am not sure about the scope of the provisioning role anymore).

flyemsafe commented 4 years ago

Yes, I think the emphasis should be on quality and core.

Given any role using this modules would depend on a specific version of the modules (until we define the module interfaces as stable, which we didn't yet), having those roles in this repo would ease saying "this role works with this module". It will also make easier to test if changes to modules break the roles.

But to answer the question raised by @Fobhep yes, collections can depend on each other, and the Ansible tooling will install dependencies automatically for you.

And yes, a staging/incubator area might be interesting.

I (personally) would like to provide opinionated roles/workflows for the following usecases:

  • katello_configure_manifest -- create a manifest on the RH portal, download it, upload it to katello. This obviously is only useful for people using RH content.
  • katello_configure_content -- either enable RH repos or add CentOS (or Debian?) products/repos, add sync plans, create content views, LCEs. Preferably in a templated way, so that the user can say: I want CentOS 7 (product/repos), a CV containing that, and Test/QA/Prod LCEs.
  • katello_manage_content -- do syncs / publishes / promotes, preferably driven by a similar input as the flow before
  • foreman_configure_network -- domains, subnets, etc
  • foreman_configure_provisioning -- compute resources, TFTP etc

All of these should aim at simple, lab-like environments (so no crazy CCVs with 100 CVs in them, or 50 subnets with 3 domains each). I think that would suffice for 80% of users. The other 20% can fork/use the roles as examples. (having written that, I am not sure about the scope of the provisioning role anymore).

This makes a lot of sense to me and it's what I attempt to accomplish here https://github.com/theforeman/foreman-ansible-modules/issues/558. To expand on katello_configure_content, a RH user can say we just purchased Red Hat Virtualization. The should be able to specify just rhv or rhv42 and the following steps get done [1]. [1] https://github.com/flyemsafe/my-satellite-post-config/blob/master/products/rhv4u2_content

bagasse commented 4 years ago

100% with @evgeni. For me the goal of this repo is to provide a way to configure foreman/katello instance from the ground up, and manage its configuration. Eg: As a foreman/katello user, I want that all my users have a restricted role that can only manage hosts via foreman interface/api, all other resources (subnet, hostgroups, compute resources, content...) will be managed via foreman ansible roles and controlled via git repo / config pipeline (it's mostly already the case today, but I was thinking to transform our playbooks to roles and make it more generic). We already have a lot of basic brick, now I think we have to provide opinionated roles for all common uses cases . Maybe we can use the foreman menu categories to organize roles (monitor, content, host, configure, infrastructure, administer).

On the staging/incubator part, it would be nice to test/share potentially future "official" roles, and maybe this kind of promotion will be ok for @ekohl and @mdellweg (depending of course on the promotion workflow, that we have to define if we agree to going on this path)

ekohl commented 4 years ago

I'm not strictly against roles/playbooks in this module but I'd like to avoid those which can have a lot of discussion because everyone sees them differently. I'd also like to avoid very specific workflows that only a few people use. That's something we see in forklift and makes maintenance hard.

I would prefer to get this repo to a 1.0.0 release without playbooks/roles and develop them in a separate collection that can depend on this. That means it can iterate quickly without thinking too much about backwards compatibility.

ekohl commented 4 years ago

Oh, and just to clarify: that's a weak preference and for some cases I see an absolute benefit. katello_configure_manifest sounds like there's very little variance in how it's used besides a few parameters. As always, it's not black and white :)

ehelms commented 4 years ago

@flyemsafe For me, your example of a user being able to configure all things needed for RHEV content is a good example of users adding value outside of the main collection through other methods such as variable files. I think these are an interesting concept, and can help drive the generic design. I would look at it like this:

If FAM supplies a katello_content_configure role, can I define a variable set that a user can combine with FAM to enable a use case.

The example of RHEV is good to have (feel free to add more to the conversation) to enable developing the generic role, and helping to determine if we can build a role that is generic enough to handle a real world scenario. If the answer to this latter part is no, then we should re-think our idea around role support.

flyemsafe commented 4 years ago

@flyemsafe For me, your example of a user being able to configure all things needed for RHEV content is a good example of users adding value outside of the main collection through other methods such as variable files. I think these are an interesting concept, and can help drive the generic design. I would look at it like this:

If FAM supplies a katello_content_configure role, can I define a variable set that a user can combine with FAM to enable a use case.

The example of RHEV is good to have (feel free to add more to the conversation) to enable developing the generic role, and helping to determine if we can build a role that is generic enough to handle a real world scenario. If the answer to this latter part is no, then we should re-think our idea around role support.

I am assuming FAM means, foreman-ansible-modules? So, this can be applied to not jut RHV, but all our layered products that are RPM based, additional examples:

By layered products, i mean at a minimum to setup a content view to install any of the above, the content view must include RHEL, in addition to the product specific repo.

Then we have or bundled solutions such as Red Hat Cloud Suite, which includes:

I would be nice for our customers who purchase that bundled could just execute a role that would create CV with all all those products.

evgeni commented 4 years ago

Okay, let me try to summarize the discussion a bit.

I think the TL;DR version is: Let's allow adding roles to this repository (and this collection) if the roles adhere to a set of (yet to be defined) rules and are small/atomic enough to be easy to use.

The longer version is probably:

please :+1: if you agree :)

evgeni commented 4 years ago

@flyemsafe I think the layered product examples are great and we could actually make layered roles out of them, so have a generic katello_content_configure role, that just configures content based on a list you feed it, and a katello_content_rhv one that uses katello_content_configure and feeds the RHV specific repos to it.

One thing that I dislike, but don't have a good solution for: katello_content_rhv would be very Red Hat product specific, which IMHO doesn't really belong into this repo. Maybe we need an accompanying satellite_content collection somewhere?

sthirugn commented 4 years ago

@flyemsafe I think the layered product examples are great and we could actually make layered roles out of them, so have a generic katello_content_configure role, that just configures content based on a list you feed it, and a katello_content_rhv one that uses katello_content_configure and feeds the RHV specific repos to it.

One thing that I dislike, but don't have a good solution for: katello_content_rhv would be very Red Hat productspecific, which IMHO doesn't really belong into this repo. Maybe we need an accompanying satellite_content collection somewhere?

I like the idea explained in this example. Create a generic role that feeds from Red Hat Product specific roles. +1 to hosting the product-specific roles in a separate project like `satellite_content1.

flyemsafe commented 4 years ago

@flyemsafe I think the layered product examples are great and we could actually make layered roles out of them, so have a generic katello_content_configure role, that just configures content based on a list you feed it, and a katello_content_rhv one that uses katello_content_configure and feeds the RHV specific repos to it.

One thing that I dislike, but don't have a good solution for: katello_content_rhv would be very Red Hat product specific, which IMHO doesn't really belong into this repo. Maybe we need an accompanying satellite_content collection somewhere?

I understand. I work in the downstream, I want our customers to continue derive value from their subscriptions.

evgeni commented 4 years ago

I've created issues for the open discussion points in https://github.com/theforeman/foreman-ansible-modules/milestone/5 and thus closing this one for now.