daenney commented 8 years ago

RFC: Support Puppet 4 only

This is going to be a bit controversial because I'm raising it this early. This is not a proposal to move to Puppet 4 only now now now. Instead it is meant for us to find a good way on how to move to Puppet 4 and define some metrics at which point we deem it acceptable to move to a Puppet 4 only version of our modules.

Reasons for moving to Puppet 4

Puppet 4 brings a vast number of improvements but besides the runtime speedup it's mostly language features and sanitisation that we, the developers, can benefit from allowing us to write more concise, bug-free, testable and compact manifests. This in turn raises the quality of our modules which is good for everyone that uses them. Some of the features deemed most important:

the type system (and thus being able to avoid all the validate_* and is_* functions) which gets us decent error reporting for free
native iteration avoiding the need for all kinds of "creative" iteration hacks (and therefor no more create_resources too)
the protected, read-only, $facts hash (instead of all the $::facts) which will facilitate migrating to a strict_variables enabled future

Additionally a new way to deal with data in modules is starting to emerge in Puppet 4 that in due time we'll be able to take advantage of too (though perhaps not before Puppet 5 depending on when the feature is deemed finalised).

Collecting data

The most important thing to do in order to decide when the time is right is to collect as much data as we can on Puppet 4 deployments and their ratio to Puppet 3.

Data that could help us make this decision:

Internal survey with Vox Pupuli members to figure out how far along our "avant-garde" is
Public survey (join forces with Puppet Labs) to get some deployment figures on Puppet 4 (perhaps every half year?)
Download statistics from the official tarballs and repositories, both FOSS and PE
Download statistics from the Forge (if they can tell us anything wrt Puppet 4 deployment)
When to move

The problem with all of this is once we move to Puppet 4 only the code will simply not work anymore on older versions of Puppet, most notably Puppet 3 which is still widely deployed. Though Puppet 4 has been out for a year now and multiple community members are holding talks at PuppetConf, Puppet Camp's and Configuration Management Camp on how to migrate to Puppet 4 and what new features it brings adoption is still slow. Puppet Labs has successfully been pushing Puppet 4 through PE upgrades.

As such we need to define a threshold at which point, based on the data that we decide to collect, we believe the time is right to move to Puppet 4. However, when we do decide to do so we should still provide critical bug fix releases for Puppet 3 compatible versions of our modules for a number of months to not unnecessarily complicate other people's lives.

Therefor I would propose a transition somewhere along these lines:

An announcement goes out through all our channels a month before this change goes into effect to give everyone some time to prepare (pin version requirements etc.)
A Puppet 3 release of the module must be available which is "perfect", i.e it doesn't generate deprecation warnings or other problems when run on the latest version of Puppet 3 with and without the future parser enabled
A major version release for every module needs to happen (preferably paired with some validation cleanup by leveraging the type system instead) and accordingly reflected in metadata.json
- _suggestion_: We should try and keep other module changes to a minimum during that upgrade. Though a major version upgrade would be traditionally where we change the API it's going to be hard enough with the language transition already that we shouldn't add too much baggage.
For a period of 3-6 months, at the discretion of the module maintainers, bug fixes and releases for Puppet 3 compatible versions of the module will still be made but no new features will be accepted

The one question that remains: how do we decide it is time?

List of tasks / bugs that we need to do/fix prior to being able to move

[ ] Puppet 4 (future parser) style guide
puppet module tool
[ ] PUP-5688: respect version constraint in metadata.json
[ ] PUP-5689: allow specifying the version of the module that should be installed
rspec-puppet
[ ] https://github.com/rodjek/rspec-puppet/issues/344
puppet-lint
[ ] https://github.com/rodjek/puppet-lint/issues/451
[ ] https://github.com/rodjek/puppet-lint/issues/450
[ ] https://github.com/rodjek/puppet-lint/issues/456

electrical commented 8 years ago

I've been asking my self this question as well for a while. The only decision I've made so far that the next major version of my modules ( 2.x ) will support puppet 4.x only. This does mean that i have to maintain 2 branches of the module and port changes back and forth. But the idea im playing with now is that when i release 2.0.0 that 1.x will be in maintenance mode.

roidelapluie commented 8 years ago

It will be time when there will be a decent linting/style guide for the new puppet 4 features, otherwise that will be horrible.

daenney commented 8 years ago

It will be time when there will be a decent linting/style guide for the new puppet 4 features, otherwise that will be horrible.

That's a very good point. Having a functional style guide will be important to avoid all kinds of unnecessary discussion on that topic.

roidelapluie commented 8 years ago

Should we make a Puppet-4-Style-Guide repo ?

daenney commented 8 years ago

It's a thought though I would suggest we post the question to puppet-dev@ first. There might already be things in the work and it would be a shame to duplicate such efforts.

danzilio commented 8 years ago

I wish PMT could actually consume the metadata in modules, i.e. installing only compatible versions. Maybe we should contribute this feature?

daenney commented 8 years ago

I wish PMT could actually consume the metadata in modules, i.e. installing only compatible versions. Maybe we should contribute this feature?

That's almost impossible to do. The master can be on a much newer version than the agent and even have modules in its environment that don't work with specific distros that the agents can be running on. The only place you could reliably do it is in a masterless environment because there you only have to care about the local machine.

danzilio commented 8 years ago

That's almost impossible to do.

grumble grumble reality grumble grumble

daenney commented 8 years ago

Alright, total aside here and stop derailing my RFCs already @danzilio but:

If there was some way of declaring API compatibility, like API levels in Android, the agents could do this work.

Consider a module which declares that it will only work with Language level 4 and is determined to be part of the catalog that an agent needs to apply which only supports up to Language Level 3. There's nothing to stop us from halting execution right then. Similarly we can probably have the agent check the module constraints, so that it doesn't try to run on a too old version of the agent or a platform that's not supported (potentially with a flag to override that if you're on one of those weird distros that usually works but no one bothers to check for).

I think what you're suggesting is actually quite feasible, just not by leveraging the PMT.

danzilio commented 8 years ago

@daenney keeps taking the bait ;)

ahpook commented 8 years ago

[PMT consuming module metadata is] almost impossible to do. The master can be on a much newer version than the agent and even have modules in its environment that don't work with specific distros that the agents can be running on. The only place you could reliably do it is in a masterless environment because there you only have to care about the local machine.

I suppose that's true, but given an agent/master workflow where you use PMT on the master to download modules, the majority case is that your agents are not going to cross major-version boundaries. One concrete improvement to PMT that might help a lot would be adding the ability to fall back to other versions if the latest/current one expresses incompatibility with the version of puppet. (Or hell, even just display what other versions are available)

ahpook commented 8 years ago

One meta comment @daenney: is the scope of this discussion about the contents of this repo? Or more broadly about stuff inside the vox pupuli namespace?

daenney commented 8 years ago

@ahpook The latter, the contents of this repo are irrelevant. We use the "plumbing" repository for things w.r.t to how work/function, the "plumbing" of our community/project.

nibalizer commented 8 years ago

We had a similar conversation on the dotnet module: https://github.com/voxpupuli/puppet-dotnet/pull/9

nibalizer commented 8 years ago

Oh man can fixing the PMT not be part of scope here?

daenney commented 8 years ago

Oh man can fixing the PMT not be part of scope here?

Completely agree. Part of a different discussion. This is part of "how do we decide when it's time to move and how do we go about it".

nibalizer commented 8 years ago

@daenney I disagree that we should drop puppet3 support before enabling puppet4 support. I think we need to have branches.

I think every module should have a table like this in the readme: https://github.com/puppetlabs/puppetlabs-stdlib#version-compatibility

If we hard cut modules to puppet4 at some arbitrary date, we'll just grow uncontrolled forks of people patching the last puppet3 version. If we keep a puppet3 branch around for a couple years, then people who haven't upgraded yet have a place to collaborate.

Creating a puppet4 branch (or moving master to puppet4) enables people to start using puppet4 and will help drive puppet4 adoption. We should do that sooner rather than later.

daenney commented 8 years ago

If we hard cut modules to puppet4 at some arbitrary date, we'll just grow uncontrolled forks of people patching the last puppet3 version.

I never proposed just yoloing it on some arbitrary date. I set very specific data sources that I would like to poll to get a feel at which point we can make this switch, where we stop actively supporting Puppet 3, without hurting people and a period in which we should continue to maintain a compatible version:

A Puppet 3 release of the module must be available which is "perfect", i.e it doesn't generate deprecation warnings or other problems when run on the latest version of Puppet 3 with and without the future parser enabled

For a period of 3-6 months, at the discretion of the module maintainers, bug fixes and releases for Puppet 3 compatible versions of the module will still be made but no new features will be accepted

Moving on.

If we keep a puppet3 branch around for a couple years, then people who haven't upgraded yet have a place to collaborate.

That's a good idea though "a couple of years" feels like stretching this a bit too far. Especially since at some point all of us will likely have moved on and needing to keep reviewing and maintaining Puppet 3 compatible code will be a drainer. At least setting the expectation that PRs will likely take a long time to be reviewed and merged if you come up with a Puppet 3 fix should be made clear.

Creating a puppet4 branch (or moving master to puppet4) enables people to start using puppet4 and will help drive puppet4 adoption. We should do that sooner rather than later.

Not unreasonable. But then we need to iron out a few things like how do we deal with a potentially diverging feature set and the fact that some of our tools are currently half-functioning on Puppet 4 like puppet-lint and spec-puppet.

We also can't just move "sooner rather than later" if the majority of our own contributors are still on Puppet 3, hence the suggested internal survey to figure out how far along we are.

ahpook commented 8 years ago

Sure, sorry to derail it @nibalizer.

I totally think the stdlib model is a workable one - major versions to express breaking compatibility with older puppet major versions. Keep a branch per major in the main repo. For the Puppet projects, we put new features only in the latest major branch but backport bugfixes as long as it's not super invasive, but YMMV.

The tooling ecosystem problem is a big one. Is there a hit-list of things needed for puppet-lint and rspec to fully work on Puppet 4?

daenney commented 8 years ago

Nope, but that's a great idea. I've added a few now.

roidelapluie commented 8 years ago

What should we do for the forge?

I think if we use branch we should also publish on the forge under different names?

puppet-corosync and puppet-corosync4 ?

danzilio commented 8 years ago

:sob: :sob: :sob: :sob: :sob: :sob: :sob:

electrical commented 8 years ago

@roidelapluie Why not just have a major version bump which is then puppet 4 ready? like have version 1.x for puppet 3.x and 2.x for puppet 4.x ?

roidelapluie commented 8 years ago

Because users rely on the Forge API "current_release" ? is PMT able to retrieve the latest release that is available for your puppet version?

daenney commented 8 years ago

That's going to cause major confusion. Is puppet-corosync4 version 4 of the module, compatible with version 4 of corosync, compatible with Puppet 4? And when we move to Puppet 5 which won't have a language transition like this, do we stay with 4, does it become 5 or do we decide to drop 3 and go back to the bare name?

This should be fixed in the PMT, it should simply be possible to instruct it which version you want fetched just like r10k and librarian-puppet can and/or respect the Puppet version constraint.

roidelapluie commented 8 years ago

ok makes sense

ahpook commented 8 years ago

This should be fixed in the PMT, it should simply be possible to instruct it which version you want fetched just like r10k and librarian-puppet can and/or respect the Puppet version constraint.

I think that's what I suggested last week :)

daenney commented 8 years ago

@ahpook I just added two issues for you related to the PMT ;).

jyaworski commented 8 years ago

So we're going to maintain two versions of puppet for our modules? That's a tall order. I would say we start with more puppet4 testing. For example, adding 4.3.1 to the test matrix in modulesync and making sure it at least runs. We can worry about porting all the syntactic sugar later.

daenney commented 8 years ago

So we're going to maintain two versions of puppet for our modules? That's a tall order.

Where did you get that from? So far, these are proposals. No road has been taken or any decision made as to what we'll support, how or when.

For example, adding 4.3.1 to the test matrix in modulesync and making sure it at least runs.

That's already the case: https://github.com/voxpupuli/modulesync_config/blob/master/config_defaults.yml#L8-L18

jyaworski commented 8 years ago

Right; it was mainly a straw-man taken from some of the earlier comments. Let me be clear with my proposal, then.

I propose we get our modules in such a state that they compile cleanly on puppet 3.8 and puppet 4. I'm fine with dropping support for <3.8.

roidelapluie commented 8 years ago

I'm fine with dropping support for <3.8 and (>= 4.0 <= 4.3)

drop 3.0 -> 3.7 and 4.0 -> 4.2

jyaworski commented 8 years ago

So how about we drop our testing for <3.8, and add explicit testing for the latest stable versions of 4.x? 4.0, 4.1, etc.

daenney commented 8 years ago

4.0 and 4.1 of Puppet 4 are unusable and not compatible with 3.8 + future parser in a number of places. 4.2 is the lowest we can safely and sanely support. Everything you're suggesting however is already in place as we always pick the latest 3.0 release.

jyaworski commented 8 years ago

Again with the -> syntax. Ignore me. :)

Keep doing what we're doing. Maybe test 3.8 with both the current parser and the future parser, just to be pedantic?

daenney commented 8 years ago

That is probably a good idea. Feel free to send a PR to voxpupuli/modulesync_config to add that one to the matrix.

ahpook commented 8 years ago

Hey all, resurrecting this. I would like to see us get to resolution on this so I can publish some docs on the front page of the Forge for all module authors who are interested in Puppet 4 support. What do y'all think about this?

Use semantic versioning for your module
Create a VCS branch for your module's existing "major" series, and express Puppet compatibility with Puppet<=3.x, Puppet Enterprise<=3.x. This will allow you to continue maintenance releases on the current series, so your 3.x users are not abandoned
Use your 'master' branch for Puppet 4-only development; express Puppet version compatibility with Puppet>=4.0, Puppet Enterprise>=2015.2. This will prevent the puppet module tool on older versions of Puppet from installing an incompatible version. You can then begin using iteration, the type system, etc without needing to support both parsers in one branch of code.

Caveats:

users who are on 3.x but using the future parser may technically be able to use your 4.x branch but the module tool will not permit downloading it (this is acceptable IMO as we want them upgrading!).
You will have to maintain two branches until you decide to drop 3.x support, but we have successfully used this model with puppetlabs-stdlib and it's not a bad way to go.

roidelapluie commented 8 years ago

Hi @ahpook

What about the style guide/best practices? Does someone already bootstrap that effort? We really need something before pushing puppet 4 only code.

daenney commented 8 years ago

Do we really though? We did without a style guide on Puppet 0.x - 2.7 and yes it resulted in some unfortunate things but the style guide as it exists now is mostly a reflection of the best practices that came out of the community.

Just about all of it still applies to Puppet 4 too. The only major thing that needs some thinking is how to deal with long(ish) type annotations, especially on parameters.

ahpook commented 8 years ago

Good point @roidelapluie I am working on that as well - but note this is not specifically about using any new language features, just making code that might not work (or might work differently) on 3.x.

roidelapluie commented 8 years ago

@ahpook is there anything public ? How van we help?

roidelapluie commented 8 years ago

@daenney no, there are a lot of other stuff.

daenney commented 8 years ago

@daenney no, there are a lot of other stuff.

How about some examples then of what you feel is lacking instead of "no you're wrong"?

roidelapluie commented 8 years ago

When to enforce type checking (sometimes a variable is type checked in the params, should you enforce it again in the «with» functions or not, etc)
How deep should we type the variables?
Should be allow the use of the «Any» data type
good/bad use cases for the reduce function (e.g: do not create resources inside a reduce function)
good bad usecase of splat array
Should be ensure String minimal length all the time (String[1] instead of String)
splat parameters: should we use

class {
  'a':
    * => $a,
    * => $b,
}

or

class {
  'a':
    * => $a + $b,
}

And there are others that I would be happy to discuss if there was a repository to do so. Maybe we can discuss that in the current style guide

hlindberg commented 8 years ago

@roidelapluie It would be very good to start a shared google doc to collect all the stylistic issues; starting with asking all the questions (like you just did). I would like to see a much more comprehensive style guide than what has been available historically and that has a bit more nuance than just "do / don't". I am happy to help out working on that.

hlindberg commented 8 years ago

In Puppet 4.5.0 there will be a --strict flag (settable to off, warning, and error). Behind that flag there will be several added new validations intended as helping maintaining a sound code base. We have an epic in Jira called "Theme: Validation" from which you can navigate to "4.x Validation", "5. Validation". Validation related tickets are associated with those. Things we could do in puppet that you think are helpful in making it easier to transition to 4.x (in the validation domain) are very welcome.

roidelapluie commented 8 years ago

https://etherpad.openstack.org/p/Puppet_4_StyleGuide

rnelson0 commented 8 years ago

We are approaching the EOL date for Puppet 3 now, just 4 months away. It appears that all of the outstanding issues are still outstanding. Are they all actually required to swing to P4 support only? I think the PMT ones specifically can be moved to a "Desired/Optional" section. Of the rest, I would like to address the puppet-lint tasks sometime between now and the end of the community summit/hack at PuppetConf and have put them all against the 2.1.0 milestone. The style guide has been receiving regular updates and I believe is future parser compatible - and the future of puppet-lint is to line its checks up with the guide. That leaves the rspec-puppet issue, which I cannot speak to.

I do think that "once we move to Puppet 4 only the code will simply not work anymore on older versions of Puppet" is not fully accurate - we can support only Puppet 4 but much of the code will continue to work unsupported with Puppet 3, until actual changes are made that rely on P4 only constructs. Only the modules that are currently Puppet 4-only would not be backward compatible on Day 0 of the change. While that may (rapidly?) change, it eases the transition.

For a timeframe, I suggest that we announce our planned drop of Puppet 3 support by November 1st via the website, twitter, and the mailing list. By November 30, we ensure that all modules have an up to date release supporting P3. Starting on January 1, we update modules to remove the support for P3 (metadata, travis, anything else?) and release new MAJOR versions when the next fix/feature is implemented. We support P3 only for emergency fixes on the older branches through 4/30/2017, one third of the calendar year.

By announcing deadlines for ending P3 support and patching ahead of time, we can ensure that it's really difficult for anyone to be surprised by the change, and we do not need to rewrite all the modules top to bottom, either.

daenney commented 8 years ago

For a timeframe, I suggest that we announce our planned drop of Puppet 3 support by November 1st via the website, twitter, and the mailing list. By November 30, we ensure that all modules have an up to date release supporting P3. Starting on January 1, we update modules to remove the support for P3 (metadata, travis, anything else?) and release new MAJOR versions when the next fix/feature is implemented.

From my experience, December and January are hectic enough with last-minute oh-no-vacation oh-shit-new-legislation god-damn-not-this-thing-too issues that we shouldn't compound that with Puppet module changes since it's so difficult for people to avoid upgrading across major version boundaries with the tools we currently have in our ecosystem (most notably PMT and nothing does any kind of Puppet compatibility checking even though the modules provide the necessary metadata).

rnelson0 commented 8 years ago

When do you think would be a better time to make the cut?

voxpupuli / plumbing