Change `drupal` to be alias for latest drupal version. Introduce `drupal11` type

rfay commented 1 month ago

Is your feature request related to a problem?

In a recent DDEV release we introduced the drupal project type for Drupal 8+.

It relies on inspection of the code laid out to determine the version, which means it's not easy to specify the Drupal version at ddev config --project-type=drupal time, except with ddev config --project-type=drupal --php-version=8.3. (And PHP version may not be the only problem in the end. For example, the inspection done in ddev config --update is used to set corepack_enable as well, for Drupal 11, and also determines the use of an appropriate version of Sqlite3.)

There are various complaints about this though, mostly because in the current setup, ddev config --project-type=drupal may result in a PHP 8.2 project (which should then be changed to 8.3 with ddev config --update). But people aren't doing the ddev config --update because it's not in their habits. (The next major release of DDEV will change most defaults to PHP 8.3, eliminating the biggest problems, but there are still complaints about how this is set or queried.)

Discussion is welcome about how we could:

Specify the Drupal sub-version at ddev config time while maintaining the generic drupal project type (or should we abandon drupal and just go back to specific versions like drupal10, drupal11, drupal12 ?
Inspect a project to find out its sub-version (ddev describe improvement?)
Other solutions?

We can probably do something about this for DDEV v1.24.0

rpkoller commented 1 month ago

i would still suggest either to automatically run ddev config --update after ddev composer create is finished or at least trigger a prompt asking the user if a config update should be run. that way the described problem couldnt happen anymore .

rfay commented 1 month ago

i would still suggest either to automatically run ddev config --update after ddev composer create is finished or at least trigger a prompt asking the user if a config update should be run. that way the described problem couldnt happen anymore .

In general, ddev config --update is too intrusive, but I agree that in this case it would be reasonable.

However, in the current timeframe with current setup, this wouldn't be an adequate solution (if you were trying to get drush installed before update) because while drupal/recommended-project installs fine with PHP 8.2, drush/drush does not.

rpkoller commented 1 month ago

In general, ddev config --update is too intrusive, but I agree that in this case it would be reasonable.

it hasnt do be shown to the user that ddev config --update is run, that could happen under the hood? and i guess running the config update wouldnt do any harm anyway if nothing changes? and if there is something that needs to change it would be more than welcome?

However, in the current timeframe with current setup, this wouldn't be an adequate solution (if you were trying to get drush installed before update) because while drupal/recommended-project installs fine with PHP 8.2, drush/drush does not.

why wouldnt that be an adequate solution? i mean if the config update would happen as part at the end of the composer create command that sets up drupal/recommended-project, at the time the user wants to require drush/drush the environment is properly set up for the task? so no php 8.2 but php 8.3. for example?

hanoii commented 1 month ago

a ddev config --update with a php upgrade would also need a ddev restart. I personally prefer the specific versions with sensible defaults, than having to rely in an obscure command. In any other platform one would still have to do this manually (update php, update composer, etc..) so why not this case. I also think that the fact that ddev config --update needs the core to be already installed. I wasn't expected that one could update drupal on php 8.2 (assume you can based on what @rfay said above), but this could not be the case in future combinations of core versions/php.

I wonder why this has never been a problem with the other frameworks that doensn't have a version specific handling.

What about supporting both? the general drupal and a more specific one where one could define it and that would validate/configure defaults before even updating anything. Like ddev config --project-type=drupal11 --php=8.2 would fail

Maybe add it as kind of tag-based format like --project-type=drupal:11 a bit more like docker images

rfay commented 1 month ago

I wonder why this has never been a problem with the other frameworks that doensn't have a version specific handling.

TYPO3 has been the primary one that does this (but it doesn't rely on any update), never had any complaints.

Maybe add it as kind of tag-based format like --project-type=drupal:11 a bit more like docker images

That's an interesting proposal.

I even think that perhaps drupal was a mistake and we should just go back to explicit project types.

gitressa commented 1 month ago

I'll re-post some parts from another, similar issue:

[...] I don't agree that config --update is a great solution. It seems kludgy to me. In my opinion, setting PHP 8.3 as the default version in July to coincide with the release of Drupal 11 would have been the correct solution. PHP projects still on 8.1 or 8.2 should bear the burden of jumping through hoops, to get a correct configuration.

It feels like Drupal is being punished for doing best practices, and using the current stable PHP 8.3. https://www.php.net/downloads.php#v8.3.12

Then, the four lines I posted above your comment would be enough.

Anyway, I am looking forward to having the PHP 8.3 as the default version, I only wish it had coincided with the release of Drupal 11 in July 2024.

[...]

PROPOSAL: Sync the default PHP version of DDEV with PHP requirements of the current release of Drupal, so that these simple steps get you the latest Drupal, with the correct PHP version:

mkdir drupal11 && cd drupal11
ddev config --project-type=drupal --docroot=web
ddev start
ddev composer create drupal/recommended-project

6306

In that issue I asked:

What do you think about my proposal?

Perhaps moving forward, DDEV could try to keep default PHP version in sync with Drupal PHP requirements, so that the default PHP version is identical to the current release of Drupal? https://www.drupal.org/docs/getting-started/system-requirements/php-requirements#versions

@rfay answered:

The reason we didn't do that earlier is that we try not to change defaults except in major version releases.

All right, but what's stopping us from planning accordingly, and making a major release, when the default PHP version needs to be bumped up?

I think getting the latest Drupal 11 with config --project-type=drupal works great. I don't see why we have to abandon it? config --project-type=drupal7 should get you the correct set up for Drupal 7. If you need a special set up, nothing is stopping you from adding a parameter, like php=8.1, overriding the default set up.

gitressa commented 3 weeks ago

I just started a new project with ddev config and went through the options, which reminded me of this issue.

As I see it, most users will only occasionally run ddev config ... after a while they probably start using parameters instead, like ddev config --project-type=drupal --php-version=8.3 --docroot=web. So it's not a big deal, if there are a lot of drupal9, drupal10 etc. entries.

@rfay: I even think that perhaps drupal was a mistake and we should just go back to explicit project types.

I tend to agree. Having the drupal "umbrella" cover multiple versions, mainly to limit the number of options, was probably not the right thing to do, in the bigger picture -- it introduced too much complexity, for too little gain.

If including all supported versions of Drupal after running ddev config makes things a lot easier, I think it's the right thing to do, after balancing the pros and cons.

It would be nice if you get the latest official release (Drupal 11) if you select drupal ... But again, if it makes things a lot easier just having a drupal11 option as seen below, I wouldn't mind at all:

Project Type [backdrop, cakephp, craftcms, django4, drupal6, drupal7, drupal8, drupal9, drupal10, drupal11, laravel, magento, magento2, php, python, shopware6, silverstripe, typo3, wordpress]:

gitressa commented 3 weeks ago

The end result should be a true Quickstart, where new projects can be initiated with a few commands:

mkdir drupal11 && cd drupal11
ddev config --project-type=drupal11 --docroot=web
ddev start
ddev composer create drupal/recommended-project

PS. Actually, maybe dev config could scan the parent folder, and if it finds "drupal11" assume that version? Similarly, drupal11web could give meaning, so that ddev config, run from inside a folder called "drupal11web" would be the same as running ddev config --project-type=drupal11 --docroot=web?

rfay commented 2 weeks ago

One thing we can do here wrt Drupal 11 that's pretty easy is to automatically

Make corepack_enable: true the default (for all project types). I'll bet this does no harm
Automatically install the sqlite3 3.45 version for all project types. I'll bet this also does no harm.

stasadev commented 2 weeks ago

Maybe add it as kind of tag-based format like --project-type=drupal:11 a bit more like docker images

I like this idea, but I'm thinking about slightly different thing

What if we add a new flag here, like --project-version=11.

When you run ddev config --project-type=drupal, it will set the project version to the default, what we set in DDEV, on the initial config. (And we will update this version from time to time.)

When you run ddev config --project-type=drupal --project-version=10, it will be the same as drupal10.

This way we can keep a generic project type for drupal, which I like because it doesn't clutter up our project types. (IIRC it was the main reason why it was added in the first place.)

And when you look in .ddev/config.yaml, it will be easy to see the used drupal version.

rfay commented 2 weeks ago

Thanks. I do think we should decide about this and make a change in v1.24.0, coming quite soon.

gitressa commented 2 weeks ago

Thanks for looking at this @stasadev and @rfay. The proposed --project-version option sounds very promising, since it could provide both simplicity via a manageable number of options, yet flexibility by allowing to define a version.

I do still think the lagging PHP version is an issue:

6306

In that issue I propose that the default PHP version of DDEV and PHP requirements of the current release of Drupal should be synced.

Perhaps DDEV could plan to make a major release, to coincide with every time Drupal's default PHP version is increased? See for example [META] Make Drupal 10/11 compatible with PHP 8.4.

rfay commented 2 weeks ago

In upcoming DDEV v1.24.0 (almost certainly the next release) the default PHP version (and drupal PHP version) will be PHP 8.3.

Unfortunately, DDEV is not driven by Drupal's plans, but we have supported them very well. We have many CMSs and frameworks and communities to support.

stasadev commented 2 weeks ago

@gitressa, I think this is exactly what you want.

The only problem is that we should have only one command to set all required configs (and make it short enough for specific Drupal defaults).

We can rely on this --project-version flag to do the same thing, that is done with ddev config --update.

And we can deprecate drupal6 and drupal7 here, and remove ddev config --update ?

For all Drupal versions, by looking at

https://github.com/ddev/ddev/blob/e84b8a7a5ca7ecdc6c02968c564397acb2da051d/pkg/ddevapp/drupal.go#L372-L413

# will set --project-version=11 --php-version=8.3 --corepack-enable
ddev config --project-type=drupal --docroot=web

# will set --php-version=8.3 --corepack-enable
ddev config --project-type=drupal --project-version=11 --docroot=web

# will set --php-version=8.3
ddev config --project-type=drupal --project-version=10 --docroot=web

# will set --php-version=8.1 (I don't know why we have 8.1 here)
ddev config --project-type=drupal --project-version=9 --docroot=web

# will set --php-version=7.4 --database=mariadb:10.4
ddev config --project-type=drupal --project-version=8 --docroot=web

# will set --php-version=8.2
ddev config --project-type=drupal --project-version=7 --docroot=web

# will set --php-version=5.6
ddev config --project-type=drupal --project-version=6 --docroot=web

hanoii commented 2 weeks ago

This is pretty much the same thing as having drupal6, drupal7, drupal8, drupal9 etc which is what we had before.

My take here is that if we are going to have different defaults depending on the version of the project (drupal in this case), I prefer it to be provided than detected. The detection can still happen for warnings or suggestions, but otherwise it's a chicken and egg problem.

EDIT: and the same as drupal:6, drupal:7, etc.. I kind of prefer that as it's more docker-like which I am used to, but either way I am ok with it.

stasadev commented 2 weeks ago

drupal:6, drupal:7, drupal:8

This is only needed for the first setup or upgrade, right?

(I don't know why, but it seemed to me that to implement this logic, we would need to add new project types drupal:10, drupal:11, and so on.)

So when you use ddev config --project-type=drupal:10, it will set type: drupal and configure all Drupal 10 defaults.

In this case, it looks much easier because we only need to parse the version, and we do not need to store it in config.

gitressa commented 2 weeks ago

Thanks @stasadev! Your examples look great, and would be a big step in the right direction, improving the configuration command.

@hanoii, I agree with "... I prefer it to be provided than detected".

@rfay, I know that DDEV is not only Drupal. Taking a helicopter perspective, my point is not about Drupal actually, but about the default PHP version in DDEV lagging behind overall ...

PHP 8.4 is close to being officially released:

The PHP 8.4 release date is scheduled for November 21, 2024.

From https://www.zend.com/blog/php-8-4

I checked the status of the most popular CMSes, and some already support PHP 8.4, or it's underway:

Drupal - PHP 8.4 under way

@andypost is working heroically hard on landing this.

Joomla - PHP 8.4 support

PHP 8.4 Support Full compatibility with PHP 8.4 ensures better performance and security as hosting environments upgrade to newer PHP versions

From https://www.joomla.org/announcements/release-news/5916-joomla-5-2-0-and-joomla-4-4-9-are-here.html

WordPress - PHP 8.4 almost support

Mission statement:

WordPress aims to support new versions of PHP on the day they are released as much as possible.

From https://make.wordpress.org/core/handbook/references/php-compatibility-and-wordpress-versions/

TYPO3 - PHP 8.4 support

**TYPO3 13 LTS stable*** [...] System requirements PHP 8.2, 8.3, 8.4

From https://get.typo3.org/

Symfony - PHP 8.4 support

Install PHP 8.2 or higher and these PHP extensions [...]

From https://symfony.com/doc/current/setup.html

DDEV roadmap for default PHP version

My proposal for a DDEV roadmap going forward, on how to set default PHP version, securing that it's up-to-date:

DDEV aims at synchronizing its default PHP version with the latest official release of PHP itself and these CMS'es and frameworks, by making a major release:

Drupal

Joomla

Symfony

TYPO3

WordPress

If that plan is followed, since most of these already support PHP 8.4 (or it's close) DDEV could aim at releasing a major version with PHP 8.4 as the default version, when Drupal 11 gets PHP 8.4 support. (January 2025?)

hanoii commented 2 weeks ago

From https://ddev.readthedocs.io/en/stable/users/quickstart/

Backdrop CakePHP Craft CMS Django 4 (Experimental) Drupal ExpressionEngine Grav Ibexa DXP Joomla Kirby CMS Laravel Magento Moodle Pimcore Python/Flask (Experimental) Shopware Silverstripe Statamic Sulu Symfony TYPO3 WordPress

That is a big list! It's been a while I check those. I guess @rfay is worried about it being a support nightmare.

On the other hand, most of those CMS have a sensible php default based on their project type:

i.e (through configOverrideAction).

https://github.com/ddev/ddev/blob/e6bdf008d6f6ecbde57b861673e619bd3984539e/pkg/ddevapp/apptypes.go#L94-L98

So in reality there should be nothing wrong with defaulting php 8.4 when it's released.. but on the other side, why would be? If we still keep a certain cms-php version mapping elsewhere. Just thinking out loud here.

rfay commented 2 weeks ago

We will not default new projects to PHP 8.4 when PHP 8.4 is released. We'll continue our overall practice of keeping up with the "more stable" release, which is one behind the latest (PHP 8.3 in next major). We do this every year when the new PHP release comes out, and have been doing this for years.

Individual project types, though, can have overrides for PHP versions on new projects if those communities prefer it.

Using the most accepted and most vetted PHP version as a default is a simple approach, and has worked for years. With DDEV people can just change the version as needed.

gitressa commented 2 weeks ago

Thanks for stating the DDEV PHP version strategy @rfay.

Actually, I thought more about it during a bike ride, and realized that the proposed --project-version flag by @stasadev will solve this very elegantly. I just didn't study the example close enough previously, sorry about that.

But with a --project-version flag, when Drupal 11 gets PHP 8.4 ready (January 2025?), it's simply a matter of switching this:

  case "11": 
    app.PHPVersion = nodeps.PHP83

... to:

  case "11": 
    app.PHPVersion = nodeps.PHP84

... which then result in this:

# will set --project-version=11 --php-version=8.4 --corepack-enable
ddev config --project-type=drupal --docroot=web

So I agree that a --project-version flag would be an elegant solution.

Syntax

If a project type flag is to be used, which of these would work best, and be the optimal format? For Drupal 10:

ddev config --project-type=drupal:10
ddev config --project-type=drupal --project-version=10

For Drupal 11, it's not needed, since the default version is Drupal 11:

ddev config --project-type=drupal

stasadev commented 2 weeks ago

ddev config --project-type=drupal:10

easy to type and remember
it's not clear what type will be inside .ddev/config.yaml (We know it's drupal, but people can think why it's not saved as drupal:10)
one type setup, that is not saved anywhere.

ddev config --project-type=drupal --project-version=10

easy to see what version was configured inside .ddev/config.yaml
harder to remember
needs a new flag that will be used only for drupal (at least for now).

hanoii commented 2 weeks ago

@stasadev I didn't imply that this was not going to be stored anywhere, they would behave exactly in the same way. drupal:10 would be a project type in the yaml.

EDIT: It'just adding the concept of tags to the project-type ala docker

gitressa commented 2 weeks ago

Thanks for the list @stasadev, it's great to see the pros and cons listed like that.

Personally, about --project-type=drupal:10, I have very little need for historical clues, so having no type in .ddev/config.yaml is fine with me. The only thing on that list for me would be "easy to type and remember" :)

The question is, which solution will be hardest to implement and maintain?

stasadev commented 2 weeks ago

I didn't imply that this was not going to be stored anywhere, they would behave exactly in the same way. drupal:10 would be a project type in the yaml.

@hanoii, okay, but I think introducing a drupal type was the right move, so that's why I suggested a new flag to manage this better. Otherwise we'll have a lot of drupal:* types again.

The question is, which solution will be hardest to implement and maintain?

@gitressa, if we need to store the exact version of Drupal, I would choose the --project-version flag, so that we can have a basic logic for this and reuse it for other types of projects if necessary. (I'm just thinking what we are going to do if Laravel or another framework introduces something breaking.)

gitressa commented 2 weeks ago

All right @stasadev, better take the safe road then, and I guess use the --project-version flag solution. Either of them really are just fine.

rfay commented 2 weeks ago

I don't see any particular benefit at this point to adding a new layer of configuration. Probably we should go back to project types of drupal10, drupal11, drupal12 if the current strategy doesn't work. The current strategy has worked fine for TYPO3 for years now. It may have just been a hiccup that it wasn't good enough for drupal11.

gitressa commented 2 weeks ago

It could be @rfay ... If as a user it's possible to do the below, that would be perfectly fine ...

Scenario, January 2025: shortly after Drupal 11 gets PHP 8.4 support, and it becomes the recommended PHP version, a DDEV MR is created and committed (simply changing drupal11 PHP version from 8.3 to 8.4) making it the PHP version for drupal11.

# will set --php-version=8.3
ddev config --project-type=drupal10 --docroot=web

# will set --php-version=8.4
ddev config --project-type=drupal11 --docroot=web

As I commented a week ago, and still is my opinion:

It would be nice if you get the latest official release (Drupal 11) if you select drupal ... But again, if it makes things a lot easier just having a drupal11 option as seen below, I wouldn't mind at all:

Project Type [backdrop, cakephp, craftcms, django4, drupal6, drupal7, drupal8, drupal9, drupal10, drupal11, laravel, magento, magento2, php, python, shopware6, silverstripe, typo3, wordpress]:

rfay commented 1 week ago

Here's what we're going to do:

drupal11 becomes a type with the correct defaults drupal project type becomes a movable alias for latest drupal version.

gitressa commented 1 week ago

Sounds fantastic, I am really looking forward to this.

rfay commented 2 days ago

@gitressa would love it if you could test the artifacts in https://github.com/ddev/ddev/pull/6750#issuecomment-2489894737

It's tricky stuff!

gitressa commented 2 days ago

Sure thing @rfay, this is exactly what I had hoped for! And it works perfectly. I tried these scenarios:

ddev config --project-type=drupal --docroot=web ... and got Drupal 11 with PHP 8.3 as expected.

Running ddev config and selecting web and drupal10 I got the desired environment for Drupal 10.

Drupal 9 drupal9 also works perfectly. With this I got a pristine Drupal 9.5.11 on PHP 8.1: ddev config --project-type=drupal9 --docroot=web

I can't wait til this gets released, it will be big step forward in user friendliness. Thank you @rfay for listening to the feedback, for taking the right decisions, and making the dream real.

@rkoller: Did you get a chance yet, to give it a test spin?

rpkoller commented 1 day ago

@gitressa yes i had already tested and sent a few observations towards randy.

gitressa commented 1 day ago

Great, thanks @rpkoller!

ddev / ddev