silverstripe / silverstripe-framework

Silverstripe Framework, the MVC framework that powers Silverstripe CMS
https://www.silverstripe.org
BSD 3-Clause "New" or "Revised" License
722 stars 822 forks source link

RFC Adopting a major release policy #10345

Closed maxime-rainville closed 2 years ago

maxime-rainville commented 2 years ago

Read our Revising our approach to major release blog post before commenting on this RFC. This will help to contextualise the discussion.

This document does not include details about specific features, many dependencies or design processes (including any topics about future CMS UI/UX, high-demand feature requests, or long-standing functionality issues).

Problematic (a background)

We painted ourselves into a corner in 2019 when we foreclosed any quick Major release.

All the dependencies we rely on follow a major releases cycle that leads them to release new majors every 2-3 years.

More and more of our time is spent on upgrading Silverstripe CMS dependencies. Many of the APIs from those dependencies are directly or indirectly included in our own API. This means that we have to choose between respecting our API contracts and adding support for the latest major. We’ve broken our semver commitments over the last few years to upgrade dependencies (PHPUnit, Swiftmailer, oEmbed).

Our current approach is meant to provide certainty to project owners, but actually increases uncertainty because every few minor releases we end up shipping some breaking changes. Project owners can’t effectively plan how much work will be needed to upgrade between minors.

It also increases the amount of work we have to do, because we jump through hoops to make our breaking changes more palatable. E.g: Dual support for PHPUnit 5.7 and PHPUnit 9.5.

Broad approach

More frequent smaller major releases at regular intervals will allow us to upgrade our dependencies more effectively. This will negate the need for us to do minor releases with breaking changes because we will be able to push all those changes to the upcoming major release.

This will also benefit Silverstripe CMS project owners by enabling them to plan their upgrades with greater confidence.

Major releases will focus, in descending order of priority, on:

New features will mainly be delivered via ongoing minor releases.

Key objectives

Assumptions

Definitions

Policy proposals

Practical implication

Transition to a release policy for Silverstripe CMS 4

Questions

Release management can be a wide ranging topic with lots of implications. When replying to the RFC, please focus on these questions so we can keep the discussion on track.

Release Lifecycle

Dependency management

Definitional issue

Implementation concerns

Note about the RFC process

Comment submission deadline

1st July 2022

Related

Indicative timeline

This is just for illustration purposes. It’s not necessary what we are going to do.

Silverstripe CMS Major release policy indicative timeline

Pull request

@silverstripe/core-team

emteknetnz commented 2 years ago

Do we want to extend the “limited support” of old major's so people can choose to skip Majors? E.g.: Upgrade from CMS4 to CMS6 bypassing CMS5?

Looking at the indicative timeline chart, the proposed limited support of 1 year seems like it already allows this, e.g. CMS limited support goes to the start of 2025, the same time that CM6 is released

Is there a preferred major release cadence? What would be the rationale for your suggestion? Do we want to reduce the support period for old Majors to 1 year?

A two yearly cadence seems best to me and is my preference

A once yearly cadence seem much too fast and would add in too much upgrade overhead for all parties IMO.

An every 3 years cadence also seems somewhat reasonable, it would provide more stability so projects are likely to benefit due to the reduced ongoing upgrade cost, though that may be offset by a greater upgrade cost when it does need to be done. From my perspective as Silverstripe maintainer, a three yearly upgrade cadence may end up boxing ourselves in with the inability to upgrade fixed dependencies. The wider software community has adopted a more frequent major upgrade schedule, so we'd likely resort to the sminnee/phpunit type of forks to keep old dependencies functional.

How do we decide which dependencies are “fixed dependencies”? Do we need an objective criteria or can we do it just by fiat?

I feel we may just have to make judgement calls on a case by case basis here. Guzzle would be a good example. We recently had to upgarde from v6 to v7 for PHP 8.1 compatibility. However because it's a popular library for making HTTP requests, it's fairly reasonable to expect some projects to be using the dependency in their project code.

When does a third party library become part of the CMS API? When we subclass their class

Yes - SapphireTest is a good example, and while it wasn't quite a API breaking change for projects because of the dual support, if you wanted to use PHP 8 then you were forced to upgrade project code.

... when we expect a parameter of a specific type / return an instance of a class

Yes - By definition that's an API change, you're parameter/return type is a different class

Do we want to perfectly track Symfony releases? This would mean releasing in even years rather than odd years.

I'd say probably not. The benefits of being aligned are we get the latest and greatest. The downside is that we're missing out on a years worth of bugfixes. Symfony is a building block and you just want things to work, so I'd prioritise stability over new features here.

How specific do we want to be with “easy to upgrade”?

I don't particularly like "easy to upgrade" for a couple of reasons.

Firstly it's a subjective term, what I personally find easy, a non-technical product owner may find highly annoying. Someone will end up disappointed because we said it would be easy and their experience was different.

Secondly, I feel it steers us away from making any innovative changes. For instance if we simply called the 4.12 release 5.0, well that would be easy to upgrade, though we're greatly limiting how much can be delivered.

I think we're better off saying something along the lines of "We'll provide a clear upgrade path" and then support it with documentation and possibly code-writing tools

What does “full support” mean in the context of older majors? Do we need to clarify it more?

Possibly change the definition as follows:

“Full support” means that we accept bugs at all levels of impact and push regular patches. No new APIs. -> “Full support” means that we accept bug fix pull-requests and do patch releases. No API modifications.

Assuming PHP versions are released in November and we release support for it around March, that gives us around 3 months to add support for new PHP versions. Are we happy with that? Do we need more flexibility?

I think 3 months should be fine, though it's possibly a little tight. I was heavily involved with both the PHP 8.0 and PHP 8.1 upgrades.

PHP 8.0 was hard because we were still using phpunit 5.7 which was way out of date by that point so we needed to update everything to phpunit 9.5. If we had the major release policy proposed here this would not have been anywhere near as painful.

PHP 8.1 was even harder because of the deprecation warnings thrown when passing null to native php functions. I'd be surprised if we get something quite that gnarly again, and we did complete the development within a 3 month window. Worst case scenario is we don't support the latest PHP version at launch and have to release it during the first minor release likely 6 months later, that shouldn't be the end of the world.

Does this policy need to specify beta/alpha release period for new majors?

I think it should for the beta only. Perhaps a 1-month beta? In practice that would mean that all fixed dependencies would need to be upgraded and probably also PHP supported added, so I wouldn't want to make it too long or it takes away development time. Alpha's we should release, though I don't see much benefit in predictable timeframes given anything can change.

How do we support community module maintainers to migrate their module to the next major release? What are some of the risks if community modules can not be updated to the latest major promptly?

Main risk is that "easy to upgrade" goes further out the window, angering project maintainers. Project will resort to using forks of modules if only to add "silverstripe/framework": "^4 || "5" to the modules composer.json. Community modules with existing unit tests are easier to support as we can provide them a free GitHub Actions CI config and pull-request so that their module is tested being installed with both CMS 4 and 5 to validate things still work. Modules without unit tests will require more in the way of manual validation.

--

Asking my own question here - what Silverstripe modules are bound by this? All of the "supported" ones? Or only the core locked-stepped ones (framework, cms, asset-admin, etc), or core-locked stepped + a selected group of high used ones e.g. elemental?

jakxnz commented 2 years ago

I think we're better off saying something along the lines of "We'll provide a clear upgrade path" and then support it with documentation and possibly code-writing tools

👍🏻

jakxnz commented 2 years ago

I'm a big fan of this. Would be really helpful to have a cadence of releases, and some quality of life improvement around anticipating releases.

Do we want to extend the “limited support” of old major's so people can choose to skip Majors? E.g.: Upgrade from CMS4 to CMS6 bypassing CMS5?

Feels moot to me; like something we should respond to (rather than pre-empt) if users are leap frogging upgrades.

Is there a preferred major release cadence? What would be the rationale for your suggestion?Do we want to reduce the support period for old Majors to 1 years?

Applying Symfony's 2-year cycle as a reference class feels valid to me. Hard to say without experiencing the rate of progress. Based on how I see the framework used in the market, 2 years feels like it aligns well with perceived ROI.

When does a third party library become part of the CMS API?

When it is needed to fulfil the requirements of the framework.

When we subclass their class (e.g.: SapphireTest)?

Subclassing principles/consequences don't feel like a concern of a release policy. Seems more a concern of architecture practices.

How do we decide which dependencies are “fixed dependencies”? Do we need an objective criteria or can we do it just by fiat?

Feels like something that would best be defined as part of authoring the policy. My initial sense is a fixed dependency is anything that can't be substituted without changing the core philosophies of the framework design.

Do we want to perfectly track Symfony releases? This would mean releasing in even years rather than odd years. Definitional issue

Am I correct that this isn't actually "years apart", this is just a few months apart? If so, I can't see any immediate practical benefits to this. My personal preference is to "perfectly track." Or in real-world terms, develop framework major releases during pre-release stages of Symfony major release; because it improves the perception of framework as a leader in expertise.

i.e Do we want to be seen as "on the ball" or "a bit behind"? There are valid reasonings for either, but my personal preference is to be seen as a knowledge leader, and my personal experience is that tools that are "on the ball" give this perception.

Assuming PHP versions are released in November and we release support for it around March, that gives us around 3 months to add support for new PHP versions. Are we happy with that? Do we need more flexibility?

Given the nature of PHP features, and the complexity of demands in the climate, I would propose this needs to be in flux. The minimum PHP version could be decided by consensus amongst the core team, as per the needs of the community, with some clear signals for how decisions are determined.

Does this policy need to specify beta/alpha release period for new majors?

No, but it would be good if it gave indicative timing of "development" and "stability" windows, as well as defined practices around when and how the community will be notified of these, so they can best participate.

How do we support community module maintainers to migrate their module to the next major release? What are some of the risks if community modules can not be updated to the latest major promptly?

As per @emteknetnz 's comment; providing a "clear upgrade path" would be a great start. Module maintainers and framework users have the same requirements in this area. They both need to be able to trust that upgrading will be tennable, and they both need to anticipate the benefits and timeline of upgrading. Alonside that, we would ideally be supporting the community with awareness and engagement, to cultivate enrollment and autonomy. Anything beyond awareness, trust-building, signposting, and reference-material feels beyond the scope of what a policy and its practices can achieve.

kinglozzer commented 2 years ago

Do we want to extend the “limited support” of old major's so people can choose to skip Majors? E.g.: Upgrade from CMS4 to CMS6 bypassing CMS5?

Looking at the indicative timeline chart, the proposed limited support of 1 year seems like it already allows this, e.g. CMS limited support goes to the start of 2025, the same time that CM6 is released

The issue is that there’s no overlap. If CMS4 is supported until December 31st 2024 and CMS6 is released “Q1 2025” then there’s a period of a couple of months (ish) where you’re running unsupported software without being able to upgrade yet. Even if we release alphas/betas earlier, there’s still not much of a window in which to get upgrades done. If we do want to support skipping releases I think we’d need to offer, for example, at least 3 months of limited support for CMS4 after CMS6 has been released. Effectively that means extending CMS4 support for another 6 months.

I definitely think it’s worth considering, but it does mean a period of supporting 3 releases at once, and developers would have double the upgrade work. Worth mentioning we wouldn’t want to offer any tools/documentation for a 4 -> 6 upgrade, you’d have to go via 5.

Is there a preferred major release cadence? What would be the rationale for your suggestion?Do we want to reduce the support period for old Majors to 1 years?

I don’t think we should offer any less than 2 years for the support period. If we extend this to 30 months (2.5yrs) for the above “overlap” then we’d still be within PHP’s support window for the latest stable PHP version at release. If we extend to 3 years of support, then developers may have to upgrade their PHP version before the end of the support period - not necessarily a problem, just an observation. Given this and Symfony’s release cadence, I think either the proposed release policy or a very similar one with a slightly extended support period would be best.

Do we want to perfectly track Symfony releases? This would mean releasing in even years rather than odd years.

I don’t think this matters too much, looking at the 6.x release line if we’d release CMS5 in Q1 2022 we’d be using Symfony 6.0, if we release in Q1 2023 we’ll be using 6.2. Either way we’ll end up on 6.4 LTS for most of the lifecycle of CMS5, and by releasing the “opposite” year to Symfony (unless I’m reading things wrong) I think we actually gain a year’s worth of bug fixes/features.

How specific do we want to be with “easy to upgrade”?

Is this asking about the terminology, or asking how much effort we want to put into tools/docs to reach a point where we can say an upgrade is “easy” (whatever we decide that means)?

What does “full support” mean in the context of older majors? Do we need to clarify it more?

I’d prefer to communicate what people can expect during each period:

Does this policy need to specify beta/alpha release period for new majors? How do we support community module maintainers to migrate their module to the next major release? What are some of the risks if community modules can not be updated to the latest major promptly?

I’m answering these together because I think they’re linked. Clearly defined alpha/beta phases will give module authors a chance to plan ahead so that they can, if they choose, provide support for a major release on day 1. It should also aid early adoption and testing because if we have a clearly defined release schedule including alpha/beta phases then projects in development are more likely to take up those pre-release versions.

It’s hard to be precise with alpha/beta phases and how long they’ll take for obvious reasons, but I think it would be useful to define a pre-release period.

rb1t commented 2 years ago

I’d prefer to communicate what people can expect during each period:

Active development Bug and security fixes Security fixes only

👍

chrispenny commented 2 years ago

Firstly I just want to say how exciting this is! I'm really glad to see this change coming in, and I think it can (overall) only be positive for the future of Silverstripe.

More frequent smaller major releases at regular intervals will allow us to upgrade our dependencies more effectively.

Speaking for myself, the other benefit I see is that it's going to keep me more active on the modules that I support. If these PHP 8 upgrades have taught me anything, it's that I can quite easily go [years] without thinking about a module, and then when upgrade time rolls around, I'm way behind the ball as I haven't been proactive with them.

This will also benefit Silverstripe CMS project owners by enabling them to plan their upgrades with greater confidence.

100% this. Some of our enterprise clients are currently quite reluctant to pick up minor upgrades due to previous upgrades [blowing out]. At the moment it's reasonably common for us to skip 3 or even 4 minor versions.

If we can re-set expectations, and prove that minors are now [as easy as they advertise], then I think we'll have a much higher adoption rate of new releases.

When does a third party library become part of the CMS API?

IMO: As soon as a developer can interact with the CMS API in a way that would change due to a breaking dependency update.

I don't think that this needs to include the the possibility that a developer has completely overridden a method (say by extending the class and using Injector), but I think that it should include considerations around the fact that a developer could access (say) protected methods and properties through Injector.

How specific do we want to be with “easy to upgrade”?

Transparency is key. I'm not sure you need to promise that every upgrade will be "easy".

The issue is that there’s no overlap. If CMS4 is supported until December 31st 2024 and CMS6 is released “Q1 2026” then there’s a period of a couple of months (ish) where you’re running unsupported software without being able to upgrade yet. ...

I definitely mimic Lozzer's thoughts here. Some overlap would be incredibly beneficial.

EG: If we find ourselves in a position where we're having to tell someone that their version of Silverstripe is EOL in [x] months, then we really do need to be able to upgrade to the latest version. It won't go down well if we're having to say "but we can only upgrade to version [y] now, and then version [z] is out in 3 more month".

jakxnz commented 2 years ago

we really do need to be able to upgrade to the latest version.

I totally see the validity of bringing an upgrade all the way to latest.

Personally, I wouldn't anticipate leapfrogging an upgrade to be easier than stepping through sequentially. When a platform has clear upgrade pathways, the path of least resistance is to upgrade sequentially. For example, going from 1.x -> 1.99 -> 2.0 -> 2.99 -> 3.0. When the software provides upgrade pathways, stepping through versions encounters clear signposts.

Would providing overlap encourage people to shortcut progression, which increases their risks and also adversely reflects on the user experience?

Also, setting the climate to pursuade sequential upgrades makes automation more practical, because automation can "play through" a sequence of updates if the underlying design is predicated on that.

mfendeksilverstripe commented 2 years ago

I'm really happy to see this policy proposal as I think it covers a lot of the pain-points we experienced.

Do we want to extend the “limited support” of old major's so people can choose to skip Majors? E.g.: Upgrade from CMS4 to CMS6 bypassing CMS5?

My recommendation is to motivate people to upgrade gradually and avoid skipping upgrades. Skipping upgrades usually leads to a larger, more complex upgrade process which needs more effort compared to several smaller upgrades.

I also noticed that skipping upgrades leads to this strange loop:

Is there a preferred major release cadence? What would be the rationale for your suggestion? Do we want to reduce the support period for old Majors to 1 year?

Two year cadence is my preference as it represents a reasonable tradeoff between supporting old versions and avoiding too many upgrades needed to keep up.

How do we decide which dependencies are “fixed dependencies”? Do we need an objective criteria or can we do it just by fiat?

I'm not sure if there is a general rule to follow here. Probably needs a case-by-case approach.

When does a third party library become part of the CMS API? When we subclass their class

Yes, I think that's reasonable.

How specific do we want to be with “easy to upgrade”?

I think every major version should highlight areas that were changed. The ease of upgrade usually depends on amount of customisation of different areas. Some project may find it to upgrade easily but others may struggle. It's probably impossible to measure the ease of upgrade in general.

Does this policy need to specify beta/alpha release period for new majors?

I think betas are useful as projects can use it to estimate the upgrade effort. I suggest to specify the beta related information as a part of this policy.

GuySartorelli commented 2 years ago

If we find ourselves in a position where we're having to tell someone that their version of Silverstripe is EOL in [x] months, then we really do need to be able to upgrade to the latest version

In theory having this policy should make that situation a lot harder to get into - you'll know two years ahead of time when each major will be EOL.

But in reality I know it won't always work out that way - you might for example inherit a project from some other vendor who had neglected upgrading to the latest major, which would force you into this position. So from that standpoint I can definitely see how having some overlap could be a good thing.

In that case it's less about "I have chosen to skip a major version" and more "Someone else has chosen to avoid maintaining this site and now I need to clean up their mess - ideally by upgrading directly to the newest major instead of being forced to choose between having an EOL project for a few months or upgrading major versions once now and again in a few months"

GuySartorelli commented 2 years ago

When does a third party library become part of the CMS API?

  • When we subclass their class (e.g.: SapphireTest)?
  • When we expect a parameter of a specific type?
  • When we return an instance of a class?

Honestly I think all of these count, assuming the methods expecting those parameters or returning those values are accessible to project code. As soon as a project can access the third party classes through Silverstripe classes they become part of the Silverstripe API.

michalkleiner commented 2 years ago

Is my understanding correct on this?

Currently, we are holding off on major changes and doing workarounds since we don't have the next major in sight and we don't want to break semver. With the new policy adopted, we will have a new major in sight in say 1 or 2 years. It's an improvement, but for some changes that might still realistically equal to never/too far out.

So reinstating major releases and having them planned helps a bit, but is there an alternative without an actual fixed release cycle where we would remove all the weight from the major versions and iterate even faster, e.g. one a year or 'as needed' sort of thing?

Probably not feasible, but when I was rambling about not having new major releases in past I also didn't imagine keeping it as strict and as planned as proposed, but it might be the best way to go for a reliable/mature software Silverstripe CMS is aiming to be.

MichAlkhouri commented 2 years ago

This is brilliant. Having a policy provides visibility across board and in turn will help manage the client and project expectation. It will also standardise our processes, reduce dependancies and risks associated with lack of visibility. This is a step in the right direction. A review of this doc next year and the year after will give an indication if the policy needs any tweaking. I would recommend a wider conversation at Silverstripe on the implementation, from an operation perspective, of this policy and it’s indicative timeline. From testing perspective this going to make my life easier as I can have a clear discussion with the client and the project on what to expect therefor I can plan what to test. Thank you

andrewandante commented 2 years ago

So reinstating major releases and having them planned helps a bit, but is there an alternative without an actual fixed release cycle where we would remove all the weight from the major versions and iterate even faster, e.g. one a year or 'as needed' sort of thing?

I'd like to see that - eventually. I think in order to get there, we'd either need more people engaged in the development of the core modules, or to reduce the number of modules that are covered, in order to achieve a yearly cadence. Part of it too would be making sure we aren't scaring people off with rapid change - I believe in this as in intermediary step. Walk before you can run and so on ;)

sminnee commented 2 years ago

Hey all, great work on the policy, it seems well-thought out and a step in the right direction for the project.

Maxime: When does a third party library become part of the CMS API? When we subclass their class? Steve: Yes - SapphireTest is a good example, and while it wasn't quite a API breaking change for projects because of the dual support, if you wanted to use PHP 8 then you were forced to upgrade project code.

I would be inclined to define "breaking change" as "Something breaks if you swap one version of silverstripe for another and make no other changes". So, the fact that you can't use PHP 8 without making other changes doesn't violate that principle.

The reason I suggest this is that the underlying motivation of semver is safe upgrades: I can pull a new version of your package and I'm not going to have things randomly break.

That said, with more frequent majors you might simply say "we'll leave that until a major release", which is also fine.

Whatever you choose, I think the main thing is to define it.

GuySartorelli: When we subclass their class (e.g.: SapphireTest)? When we expect a parameter of a specific type? When we return an instance of a class? Honestly I think all of these count, assuming the methods expecting those parameters or returning those values are accessible to project code. As soon as a project can access the third party classes through Silverstripe classes they become part of the Silverstripe API.

I would be more restrained like this and say something along the lines of when our recommended (or implied recommended) approaches to using code mean that you interact with the 3rd-party classes directly.

Maxime: How specific do we want to be with “easy to upgrade”? Steve: I don't particularly like "easy to upgrade" for a couple of reasons. Firstly it's a subjective term, what I personally find easy, a non-technical product owner may find highly annoying. Someone will end up disappointed because we said it would be easy and their experience was different. Secondly, I feel it steers us away from making any innovative changes. For instance if we simply called the 4.12 release 5.0, well that would be easy to upgrade, though we're greatly limiting how much can be delivered.

A useful principle is "what % of the original build cost is likely to be spent on an upgrade?" it's really hard to measure so it would be more like a principle / aspiration than a commitment, but:

I don't know what the right aspiration number is... maybe 20% is more realistic? but it should probably be low enough that few organisations who care about security are left unable to fund an upgrade.

So let's say that a project that took a team of 3, 4x fortnightly sprints to build - 24 person-weeks. Could that be upgraded in 2.4 person weeks?

That would seem achievable if it was a clear step-by-step process that didn't require manually touching every line of code and didn't leave 'missing upgrade steps' 95% of the time.

There's all sorts of factors that come into play that make it really hard to predict this, but you might be able to ask project maintainers (both inside and outside SSLtd) to report back on this...

I think it should for the beta only. Perhaps a 1-month beta?

The following would be useful things to do with a beta:

In that light, is 1 month long enough? You could also do both of these against the .0.0 but will people expect that some of this has been done already?

sunnysideup commented 2 years ago

Awesome initiative

lekoala commented 2 years ago

Anything that helps moving forward seems like a great idea!

maxime-rainville commented 2 years ago

Thanks everyone for your feedback. I’ve tried to summarise the general understanding I got from all the submissions and answer some of the key questions that came up.

Major leapfrogging

With the current envisioned approach, there wouldn’t be any support overlap between non-consecutive majors. e.g.: At no point would CMS4 and CMS6 be supported simultaneously.

If you want to always be on a supported version, leapfrogging is not possible under the proposal as written. Obviously, if you don’t care about being on a supported version, you could choose to leapfrog.

Let’s say we gave people an extra 6 months of overlap, would that be enough to plan and perform an upgrade? I’m trying to imagine a client who cares a lot about being on a supported version, but doesn’t want to spend a lot of time upgrading their project. After sitting on their hands for two years, they turn on a dime and manage to pull off a double upgrade in 6 months. Honestly, that sounds very implausible to me.

I’m sure a lot of people would like to have the option in theory, but I don’t think many people would exercise it in practice. I would like to hear from people who think this would be their preferred approach to managing major upgrades … if they exists.

On the other hand, “limited support” is not very expensive for us to provide … so long as there isn’t a major vulnerability in an EOL dependency. It’s more of a question of risk management.

Release cadence

I didn’t see anyone making the case for annual majors. Everyone seemed reasonably comfortable with majors every two years. I saw some people making the case for majors every three years, but I didn’t get the impression anyone was deeply committed to this view over the two year proposal.

The main problem I see with a new major every three years is that we would have to retroactively add support for new PHP release to existing majors twice instead of once. Speaking from previous experience, adding support for new PHP releases is the thing that has been painful for us in the past. That would also increase the amount of technical debt we would build up before having a chance to pay it down.

I’m inclined to stick to the two year cadence.

So reinstating major releases and having them planned helps a bit, but is there an alternative without an actual fixed release cycle where we would remove all the weight from the major versions and iterate even faster, e.g. one a year or 'as needed' sort of thing?

I think this is somewhat the approach the PHPUnit maintainer has taken. Basically, every year, he looks arounds and makes a judgement call about whether he can support the new PHP version without doing a major.

I don’t think this model would work well for us. PHPUnit is much more mature, has a much narrower focus and has less dependencies.

At any given time, there will be things we would like to break. We have much more of a balancing act to pull off between “breaking things” and “stability”.

Dependency part of our API

The main concern I had here was for dependencies we use for internal logic, but don’t directly expose through our API. Traditionally we haven’t considered that to be part of our API and haven’t had any qualms about swapping those out. In theory we respect SEMVER when we do this.

However, it’s possible we could create an accidental conflict if the parent project also uses this dependency.

Another thing we should consider is making better use of interfaces. For example, let’s say we have a method that returns a HTTP client … the return type should say Psr\Http\Client\ClientInterface even if we always return an GuzzleHttp\Client. In my mind here, Guzzle is not part of our API and we could decide to swap out the return object of that method with some other PSR7 client.

Defining fixed dependency

I didn’t see any clamouring for an objective definition of what a fixed dependency should be. I’m inclined to just publish a list of fixed dependencies when we release a new major and leave it at that.

Symfony Major releases

If we release CMS5 in early 2023, we will ship with Symfony 6.2. Symfony 7.0 will ship at the end of 2023, but we won’t add support for it in CMS5.

If we ship CMS5 in early 2024 instead, it will ship with Symfony 7.0. By the time Symfony 8.0 comes out, it will be about time for us to release CMS6.

The second scenario is a bit more elegant. However delaying CMS5 to 2024 would be pretty painful, because we would have to consider upgrading a lot of CMS4 dependencies if we went with this approach and potentially adding support for PHP8.2.

Symfony is pretty mature and has long support timelines, I don’t really see a big advantage to be running the latest and greatest version all the time. Arguably, having a bit of lead time to give a chance for other vendors to add support for the latest version of Symfony is a good thing.

Defining “easy to upgrade”

From reading everyone’s comment, the general vibe I get is we should stay clear of the “easy to upgrade” language to avoid confusion.

@emteknetnz’s proposal seems to have general support.

“We'll provide a clear upgrade path" and then support it with documentation and possibly code-writing tools.

Defining “Full support”

@kinglozzer suggested moving away from the “full support” language and describe what kind of changes people can expect in each support phase.

I’d prefer to communicate what people can expect during each period:

  • Active development -Bug and security fixes -Security fixes only That approach seems sensible to me. We’ll probably want to align our minor release commitments to those definitions as well.

3 months long enough to add PHP support?

@emteknetnz who has been leading our last couple PHP upgrades said that 3 months is probably enough, although we might need 6 months in the worst case scenario. That sounds reasonable to me.

Historically, we haven’t put much effort on running pre-release PHP versions. If we did, we could probably get started adding support for the next major release earlier in the cycle.

My biggest question mark is how we would handle a hypothetical PHP9.0. If we had support for it in a minor release that could be unpleasant. Given our previous experience, it probably will be OK. I’m not 100% sure there’s anything we can do beyond waiting to see when PHP9.0 is released and what breaking changes are shipped in there.

Given the nature of PHP features, and the complexity of demands in the climate, I would propose this needs to be in flux. The minimum PHP version could be decided by consensus amongst the core team, as per the needs of the community, with some clear signals for how decisions are determined.

I don’t think we can do that. That used to be our defacto policy when we launched CMS4. Since then, most of the PHP world has adopted a much faster pace of major releases and keeping all our dependencies working with older PHP releases became unmanageable. Adding support for newer PHP releases almost always means dropping support for older ones. We adopted a formal PHP support policy last year. The current proposal is in many ways an offshoot of that earlier discussion to apply that PHP support policy to Major releases.

Formal alpha/beta releases

From reading the comments. I get the impression that there’s at least some appetite for betas. Our current beta period for minors is 1 months … I’m thinking it needs to be longer for majors.

A complicating factor is that we would want the beta to be supporting the newly released PHP version. So if we are aiming to release a stable major in March and have a beta version available ~3 months before end, that would literally only give us 1 month to add support for a new PHP version in a new major with the holiday break in the middle.

I’d be inclined to use more generic language like “we aim to have an approximately 3 month beta period.” If we can start working with pre-release PHP versions early on, that will facilitate this process considerably.

On the alpha questions, my thinking would be to tag them early and often. Basically, once we start actively working on the next major CMS release, we should aim to tag the alphas everytime we complete a substantial piece of work. I wouldn’t commit that in the policy however.

My current thinking at this stage is:

Support community maintainers to migrate their module to the next major

I didn’t see a lot of comments on this topic beyond saying that it’s more or less equivalent to “provide a clear upgrade path” for regular projects.

I don’t know that this should block adoption of the policy, but I would like to have a follow up conversation to make sure we get this properly covered.

Some symfony packages can actually work pretty well across majors. I’m not saying we should aim to allow people to do something like ”silverstripe/framework”: “^4||^5”, but if it’s possible to do it without doing anything horrible, I think we should aim to not foreclose the possibility.

Other comments

What Silverstripe modules are bound by this? All of the "supported" ones? Or only the core locked-stepped ones (framework, cms, asset-admin, etc), or core-locked stepped + a selected group of high used ones e.g. elemental?

My instinct right now is “all of the supported modules” with the possibility of reassessing which modules are supported before each major release.

I don’t think we should do major releases of individual modules during a major release cycle … or if we do, we should still support the older major of that module for the rest of the release cycle.

One thing to consider is that we did a few quick iterations of a few modules early into the life of CMS4: versioned-admin, elemental, graphql, mfa, login-form. Maybe that’s just the cost of innovation. We probably need to have a chat about how we deliver new features within the major release lifecycle. Maybe new capabilities start of as optional modules that we iterate over quickly … Once we decide some capability is well defined, it can be folded back into core at the next major release.

rb1t commented 2 years ago

Max et al, awesome work here. It's amazing that we can contribute (and benefit from) not only to SS's technical development, but also its strategic/policy development. Thanks for opening this dialogue.

My comments that follow are more client-centric (as well as considerations relating to managing their projects and expectations). I think it's important to maintain that state of mind here (or at least, it's particularly important for us agency folk). As much as a lot of this deeply impacts developer workflow, many of us would be out of jobs if the model isn't feasible, or sustainable for our clients.

Let’s say we gave people an extra 6 months of overlap, would that be enough to plan and perform an upgrade?

If the 6-month overlap is announced well in advance that's probably fine (i.e., in a rolling predictable roadmap, and/or one that announces the overlap months ahead). Otherwise, if the 6-month overlap period starts at the time of the new Major version's release (without committing to the proposed, predictable cadence), that could be challenging. For even if we can technically upgrade all of our client's sites in 6 calendar months, getting them coordinated can take longer (e.g., budget approvals, coordinating downtimes, coordinated interactions with 3rd party tech, data migration, etc.).

I’m trying to imagine a client who cares a lot about being on a supported version, but doesn’t want to spend a lot of time upgrading their project.

I think this is very common. In my experience, small-to-medium businesses care about updates (e.g., they understand the value of security patches, welcome bug fixes, etc.) but have difficulty committing resources. Time, yes; but more commonly, money. This is less common for large-to-enterprise orgs, but they can still end up in a similar spot, (albeit, for different reasons). Again just my experience, but for them, it's often more about the time: not wanting any interruptions to business continuity, and keeping internal resources focused on their products/services. This is actually a much bigger strategic topic that'd I'd be happy to provide more thoughts on; but the relevance here, is that if expectations can be managed clients will be more likely to be able to do upgrades more consistently (and overall, they will be happier). E.g., have more time to make it a part of their fiscal budget, mark the calendar for a content freeze, etc.

After sitting on their hands for two years, they turn on a dime and manage to pull off a double upgrade in 6 months. Honestly, that sounds very implausible to me.

I think you're correct with this assessment. And in these situations, it's probably more of a leap-frog over the Major version they neglected, and wait for the subsequent release (setting expectations that they're going to be in an unsupported version for a while). And if that's concerning, it may just mean a lot of sacrifices need to be made ... to timelines, and wallets :)

In the first year following a Major release, most of our energy would be on adding features to that new Major and maintaining the old Major … with some high level thinking around what big changes we might want to ship in the next Major release. In the second year following a Major release, the old Major enters limited support which will free us to start actively working on the next Major. As the year progresses, more and more of our efforts will shift to working on the next Major and less on adding new features to the current Major.

This makes a lot of sense. That said, I think the biggest challenge is not getting into a situation where the period of 'maintaining the old Major' isn't getting neglected while attention shifts to active dev on the next release. For example, we know it took a lot of love to get SS 4 in the good spot that it is today (e.g., UX improvements, bug fixes, etc.). Knowing 4x has been actively worked on for 5 years now (to get it in a the really good spot it is), committing to a 2 year cycle will probably require strict balancing of how many / how big the new features & architectural changes are. You don't want to bite off more than what is chew-able!

A summary of what I like in this discussion:

GuySartorelli commented 2 years ago

committing to a 2 year cycle will probably require strict balancing of how many / how big the new features & architectural changes are

One of the major considerations here is that we will continue our commitment to semantic versioning - in other words all new features will be aimed at new minor releases. With the jump from SS3 to SS4 it seems like there was a larger focus on having SS4 be new and shiny and have lots of cool new toys. The new policy proposal avoids that footgun which makes this balance a lot easier - we know ahead of time that the major release isn't about new functionality, it's primarily about updating dependencies and paying down technical debt.

xini commented 2 years ago

I am very much in favor of this change. 2 years seems reasonable. It will bring a lot more stability and predictability to Silverstripe.

Also, I appreciate the wider community involvement in this. Thanks for asking and listening!

SemVer

Sam: The reason I suggest this is that the underlying motivation of semver is safe upgrades: I can pull a new version of your package and I'm not going to have things randomly break.

This I think is THE most crucial point here. It has to be possible to upgrade minor SS releases without changing a thing on a site and setup. That includes when methods are overwritten using Injector. And it also includes not dropping support for EOL PHP versions until the next major. Only then semver really works and only then SS releases become reliable.

Release lifecycle

I think that major releases should only go into support stage once the next major has been released. Or at least the EOL clock should only start counting once the next major has been released. The reason for this is that there might be delays with next major (think for example PHP 9), which would shorten the lifetime of the previous release.

Also, I would prefer having 2 years of limited support (a.k.a. security fixes), which would give an overlap of 1 year.

emteknetnz commented 2 years ago

And it also includes not dropping support for EOL PHP versions until the next major.

I can see the appeal of that from a project perspective, though I'm not sure how viable that would be, that would be reversal of the policy adopted in 4.9.0 to drop EOL PHP versions - https://www.silverstripe.org/blog/some-upcoming-changes-to-silverstripe-cms/

Off the top of my head I can think of a few reasons not support EOL PHP versions:

I'm curious though, how much does the need to update PHP block projects from running composer update ?

xini commented 2 years ago

And it also includes not dropping support for EOL PHP versions until the next major.

I can see the appeal of that from a project perspective, though I'm not sure how viable that would be, that would be reversal of the policy adopted in 4.9.0 to drop EOL PHP versions - https://www.silverstripe.org/blog/some-upcoming-changes-to-silverstripe-cms/

Exactly. That policy is wrong IMO.

Off the top of my head I can think of a few reasons not support EOL PHP versions:

* Additional resources required to keep testing older versions. These have to come from somewhere so it will come at the expense of moving things forward.

I disagree. The pipelines are already setup and already work. There is no additional work required to just leave them there.

* Recent releases of third party dependencies may no longer support EOL PHP versions

If there are technical reasons to drop support (like for example the move to PHPUnit 9), that's totally legit. But not just for the sake of it.

* We generally encourage projects to get off EOL PHP versions and move to the latest supported version of PHP. Quite simply staying on out of date software is a poor security practice

Correct. But that's not your responsibility. That's the responsibility of the developer looking after the site.

I'm curious though, how does the need to update PHP block projects from running composer update?

If say SS 5.3 dropped support for say PHP 8.0 and the site runs on 8.0, then a composer update would not do anything and the site would not receive the bugfixes and security updates for SS because the newer version requires PHP 8.1+. Composer doesn't complain, it just doesn't do anything. That's the bigger risk than running on an EOL PHP version.

emteknetnz commented 2 years ago

Apologies, I originally worded my question wrong, I've since changed it to "I'm curious though, how much does the need to update PHP block projects from running composer update ?"

I'm mostly curious how big of an issue updating PHP on a webserver is, and how much having to do this is a blocker from running composer update i.e. just updating PHP may be more work than the code upgrade.

xini commented 2 years ago

Apologies, I originally worded my question wrong, I've since changed it to "I'm curious though, how much does the need to update PHP block projects from running composer update ?"

I'm mostly curious how big of an issue updating PHP on a webserver is, and how much having to do this is a blocker from running composer update i.e. just updating PHP may be more work than the code upgrade.

We encounter a lot of organizations where we as the site developers don't have access to the server directly. There is a lot of red tape through IT departments etc to get PHP updated. Not everyone runs their sites in containers.

We as developers can still run composer update, but it doesn't do anything in that case. We might not even notice, because composer doesn't fail.

I believe that dropping support for a PHP version is a breaking change and should need a major.

Not receiving security updates for SS and its dependencies is probably the bigger risk than running on EOL PHP.

jakxnz commented 2 years ago

I'm curious though, how much does the need to update PHP block projects from running composer update?

If I'm understanding you correctly @emteknetnz, pretty extensively. When composer update is done in a downstream state, every dependency may be subject to a new source, and every new source raises the risk of conflicts. It's pretty prohibitive without the resources to handle it.

emteknetnz commented 2 years ago

Interesting points

One possibility to align Silverstripe with the PHP timeline of non-EOL versions would be to simply never support PHP 8.1 in CMS5, meaning that

Active development for CMS5 would cease in December 2024. It will be supporting PHP8.1, PHP8.2 and PHP8.3.

Would become just PHP 8.2 and PHP 8.3. We could then retain our policy of not supporting EOL PHP versions. This should mean that a CMS5 project will never need to update PHP

This would mean a decrease in the ability install CMS5 early, as webservers will be less likely to have PHP 8.2 support as they do PHP 8.1 support. I'd be interested in other peoples thoughts on this

xini commented 2 years ago

Would become just PHP 8.2 and PHP 8.3. We could then retain our policy of not supporting EOL PHP versions.

No! If CMS5 starts with supporting PHP 8.1, then it needs to end with supporting it.

emteknetnz commented 2 years ago

Sorry, I did not word that very well. What I meant is that a possibility could be that when CMS5 is released it does not support PHP 8.1 i.e. silverstripe/framework has a composer.json with "php": ">=8.2". When PHP 8.3 is released in late 2023, then that too would be officially supported.

xini commented 2 years ago

Sorry, I did not word that very well. What I meant is that a possibility could be that when CMS5 is released it does not support PHP 8.1 i.e. silverstripe/framework has a composer.json with "php": ">=8.2". When PHP 8.3 is released in late 2023, then that too would be officially supported.

So you're releasing software in 2022 that doesn't support PHP 8.1 that still has 2 years until EOL?! That's weird too!

kinglozzer commented 2 years ago

I’ve been thinking about PHP versions - a quick example. If you’re going to put a CMS5 project live in Q1 2023, the latest PHP version will be 8.2. PHP 8.2 will be supported until November 2025 and while CMS5 is supported until December 2026, “active development” will end in December 2025. That means that even if we keep the policy of dropping support for EOL PHP versions, we wouldn’t drop PHP 8.2 until November/December 2025 - at which point CMS5 moves into security fixes only anyway.

So what we could propose is that the newest PHP version at the time of release (8.2 for CMS5) will be supported throughout the entire major release lifecycle. Once active development stops, we’re only looking at security fixes for those final 12 months so there hopefully won’t be any upgrades to dependencies that need an upgrade to PHP 8.3.

(With that proposal, CMS5 would still support 8.1 at release and drop support for it in November 2024)

The word “hopefully” is doing some heavy lifting there, so I think the caveat is that supporting an EOL PHP version would only ever be “best effort”. For example if a security issue in a dependency required dropping an EOL PHP version in order to fix it, we’d have to just do it and anyone running EOL software would be stuck without patches going forward. I don’t see any obvious way around that, and to be honest I don’t think we should jump through hoops to find one.

jakxnz commented 2 years ago

Comment submission deadline: 1st July 2022

Parkinson's law, amiright 😄?

maxime-rainville commented 2 years ago

That said, I think the biggest challenge is not getting into a situation where the period of 'maintaining the old Major' isn't getting neglected while attention shifts to active dev on the next release.

That's definitively a concern that we have to keep track of.

We haven't quite figure out how we will handle CMS4 support once CMS5 is out. My instinct is that it's probably going to be a lot easier to back port fixes from CMS5 to CMS4, than it was to back port fixes from CMS4 to CMS3.

Also, we got a lot better at delivering patch releases for individual modules since those early CMS4 days.

What I meant is that a possibility could be that when CMS5 is released it does not support PHP 8.1 i.e. silverstripe/framework has a composer.json with "php": ">=8.2"

The proposal as worded right now is that at launch, new Majors would require an actively supported PHP version. If we release CMS5 in March 2023:

Supporting only the very latest PHP version seems a bit too aggressive for me.

Support for EOL PHP version in older majors

I've put the CMS5 propose support timeline next to the PHP support timeline (rounded to the nearest year for clarity)

image

If we carry on with our policy of dropping support for EOL PHP, when CMS5 enters full support in 2025, we would drop support for PHP8.1. Then in 2026, CMS5 would enter limited support, drop support for PHP 8.2 and require you to be on PHP8.3.

I can see @xini's point that this seems weird. The reason why we put a policy of dropping EOL PHP version is so we could more easily add support to newer PHP version. But in this case, we are explicitly saying that we wouldn't be adding PHP8.4/PHP9.0 support to CMS5 ... so why would we drop support for PHP8.1/PHP8.2?

If we are only doing bug fixes to CMS5 from 2025 onward, is there a major downside to make sure those bugfixes are still compatible with PHP8.1 and to keep our PHP8.1 builds running? Probably not.

xini commented 2 years ago

But in this case, we are explicitly saying that we wouldn't be adding PHP8.4/PHP9.0 support to CMS5 ... so why would we drop support for PHP8.1/PHP8.2? If we are only doing bug fixes to CMS5 from 2025 onward, is their a major downside to make sure those bugfixes are still compatible with PHP8.1 and to keep our PHP8.1 builds running? Probably not.

I agree wholeheartedly!

maxime-rainville commented 2 years ago

Thank for all the feedback. We have open a PR to implement the conclusion of all the info we got. The PR is still a draft at this stage, but I think we can consider the RFC accepted.

https://github.com/silverstripe/developer-docs/pull/54

GuySartorelli commented 2 years ago

The major release policy has been adopted: https://docs.silverstripe.org/en/4/project_governance/major_release_policy/ A blog post will be coming to talk more about it - in the meantime I'm closing this RFC as it has served its purpose.