[draft] Change records for GovCMS9

pandaskii commented 4 years ago

Drupal core
Testing
- new testing method to distribution
Theme
- new theme to distribution
- theme deprecation plan
Modules

Drupal core

[x] Pin drupal/core to 9.1.7

Testing

new testing method for distribution

[ ] Cypress code

Theme

new theme to distribution

[x] govcms_bartik - a simple prototype theme from Drupal Bartik. This is the default theme for a fresh install site code

theme deprecation plan

[x] Set Claro as the default admin theme. You can keep using drupal/adminimal_theme
[x] Remove govcms8_ui theme from the GovCMS distribution (it is unsupported for a while)
[x] Remove govcms8_uikit_starter theme from the distribution
[x] Remove govcms_install theme from the distribution

Modules

new module to distribution

[x] drupal/bigmenu - also back ported to GovCMS8 distribution
[x] drupal/rest_menu_items - no plan to be back ported to GovCMS8 distribution
[x] drupal/migrate_source_csv - no plan to be back ported to GovCMS8 distribution
[x] drupal/migrate_tools - no plan to be back ported to GovCMS8 distribution
[x] drupal/migrate_plus - no plan to be back ported to GovCMS8 distribution
[x] drupal/migrate_file - no plan to be back ported to GovCMS8 distribution
[x] drupal/twig_tweak - no plan to be back ported to GovCMS8 distribution

Deprecated custom modules

Ref: https://github.com/govCMS/GovCMS/tree/2.x/modules/deprecated

[x] govcms8_default_content (removed - dummy module provided)
[x] govcms_content_types and sub modules (govcms_blog_article, govcms_event, govcms_foi, govcms_news_and_media and govcms_standard_page) (removed - dummy module provided)
[x] govcms_search (removed - dummy module provided)
[x] govcms_workflows (removed - dummy module provided)
[x] govcms8_foundations and its sub modules PR

Deprecated contrib modules

[ ] Remove drupal/features module from the distribution
[x] Remove module_filter from the distribution
[x] Remove drupal/memcache module from the distribution
[x] Remove drupal/permissions_by_term from the distribution
[x] Remove drupal/page_manager from the distribution
[x] Remove drupal/graphql from the distribution

Future module deprecation plan

[ ] Remove block_place module drupal
[ ] Remove config_filter and use the new API from Drupal core drupal

joshua-graham-adelphi commented 3 years ago

@pandaskii - Where can we find more information about each dot point ?

pandaskii commented 3 years ago

@joshua-graham-adelphi I will update this change records in this week to reflect the latest changes and link them to code base/PR.

Most items are tracked in our internal Jira as a start point. :) If you need more information, pls feel free to create an issue or ping me in Slack (either Drupal or GovCMS Slack)

joshua-graham-adelphi commented 3 years ago

Why are deprecated custom modules actually "removed" (dummy module - all functionality removed) ? not deprecated in the strict sense of the word? (referring to govcms8_* modules).

If this the plan - can we please update the terminology to "removed - dummy module provided" accordingly (or similar thereof).

pandaskii commented 3 years ago

@joshua-graham-adelphi I've updated the draft with your suggestion "removed - dummy module provided"

govcms8_foundations and its sub-modules will be relocated to the "deprecated" folder - it will be discouraged for the new site but the existing GovCMS8 site could still use it and hopefully, it can reduce the burden to migrate from GovCMSD8 to D9

joshua-graham-adelphi commented 3 years ago

Thanks :-)

ocean commented 3 years ago

@pandaskii I'm in the process of upgrading a GovCMS driven Drupal 8 site to Drupal 9, we use govcms_standard_page which in turn seems to depend on several of the other deprecated modules, and the database upgrade is failing, Drupal is complaining these modules aren't installed because they have been moved I think?

We are installing govcms/govcms:2.4.0 with Composer, so they are being installed, but now in that "deprecated" directory.

Do you have any suggestions for how to fix this issue??

pandaskii commented 2 years ago

@ocean what is the errors that you have?

govcms_standard_page is a stub module in GovCMSD9 now. If you don't have modifications of this module in GovCMS8, the migration shall not complain it.

joshua-graham-adelphi commented 2 years ago

Hi @ocean

Here are some commands that can helped me after moving the code from govcms 8 => 9.

printf "DELETE FROM key_value where name = 'system.module.files';" | drush sql-cli
printf "DELETE FROM key_value where name = 'system.theme.files';" | drush sql-cli
printf "DELETE FROM key_value where name = 'system.profile.files';" | drush sql-cli
drush cacherebuild
drush eval "\Drupal::service('extension.list.module')>reset()->getList();"
drush eval "\Drupal::service('extension.list.theme)>reset()->getList();"
drush eval "\Drupal::service('extension.list.profile')>reset()->getList();" 
drush cacherebuild 
drush updatedb 
drush cacherebuild

# If Drupal::service('extension.list.X')>reset() doesn't work (crashes) you can also try...
printf "DELETE FROM cache_default where name = 'core.extension.list.module';" | drush sql-cli
printf "DELETE FROM cache_default where name = 'core.extension.list.theme';" | drush sql-cli
printf "DELETE FROM cache_default where name = 'core.extension.list.profile';" | drush sql-cli
printf "DELETE FROM cache_default where name = 'system.theme.info';" | drush sql-cli
printf "DELETE FROM cache_default where name = 'system.theme.files';" | drush sql-cli

Feel free to join the slack channel and other community support channels by sending a request via govcms contact us form.

ocean commented 2 years ago

@pandaskii @joshua-graham-adelphi Thanks for getting back to me.

I've now removed all the dependencies that our custom modules had on any of the deprecated GovCMS modules (we were depending on govcms_standard_page, govcms8_foundations and govcms_media), but the during the database update step in deployment, the system now loudly complains:

[info]: Preparing database update.
 [error]   (Currently using Missing or invalid modules The following modules are marked as installed in the core.extension 
configuration, but they are missing:
 * govcms8_calendar_item
 * govcms8_foundations
 * govcms8_layouts
 * govcms8_modifiers
 * govcms_content_types
 * govcms_media
 * govcms_standard_page

Review the  suggestions for resolving this incompatibility [1] to repair your 
installation, and then re-run update.php.

I've removed them from the core.extension.yml file, but of course it hasn't got to the config import yet so it doesn't know they're not referenced any more.

Those commands look like they might be useful, thanks @joshua-graham-adelphi - otherwise I might add a script which moves the module files back into their original location, then after they've been uninstalled I can remove them again.

pandaskii commented 2 years ago

@ocean

Are you with a PaaS site and working to update the package in the existing GovCMS D8 site?

All the GovCMS D8 SaaS sites are migrated already, thus I think you could work for a PaaS project.

PIng us on Slack if needed :) A helpdesk ticket is also welcomed since GovCMS D8 is in EOL very soon.

From the error message, looks like it is a mix of config import from GovCMS D8 instance. Disable/rename the default config files folder to disable this feature, you don't need these legacy configs during the migration. Just a latest GovCMS D8 database will be fine then a new set of configs can be exported after migration.

Here is my personal approach with PaaS, just as an example here

Set a fresh D9 PaaS in local. A new PaaS scaffold is given https://github.com/govCMS/scaffold
Modify the files with extra modules/libraries etc from D8 instance. A new PaaS scaffold is given https://github.com/govCMS/scaffold
Import the D8 database, copy the modules, themes and files from existing D8 site. Don't copy configs, there are a few changes from D8->D9
Run db update and troubleshoot from here
Uninstall the stub/deprecated modules from GovCMS D9

To test a fresh install GovCMS D9 PaaS site using lagoon in local environment:

git clone https://github.com/govcms/scaffold paas-test
cd paas-test
Run “ahoy init paas-test paas 9”
Run “ahoy up”
Run “ahoy run composer install” - it can fix the “no input file” issue in local environment
Run “ahoy install” - first time to install the site
Run “pygmy up” if it is not up
Visit http://paas-test.docker.amazee.io - this URL is defined in .env file

ocean commented 2 years ago

@pandaskii Yep, I'm working on a PaaS site, My Aged Care.

On Friday I finally got the deployment running, by copying the deprecated modules back into their previous filesystem location Drupal was then able to run the DB update without breaking and then the config import uninstalled those modules.

I now just need to some branching logic to not perform the copy if the modules aren't installed any more.

Your steps above is pretty much what I did, the D8 -> D9 upgrade (with upgraded config yml) was working fine on my local but the DB update was being blocked when running in Lagoon, not sure why I didn't have the same problem locally. Now I'm going to run another test doing an upgrade in a test environment with a fresher dump of our production DB, will see how that goes.

I've tried to get onto the GovCMS Slack but didn't get a reply after going through Contact Us. I am in the Drupal Slack though and have seen you there too :) Will shout out if I need some more support, but think I'm alright now.

Thanks for your help 👍

joshua-graham-adelphi commented 2 years ago

Hi @ocean,

Okay, seems strange that you need to copy the modules. I suggest, performing the upgrade locally and re forklifting the site if you have issues in lagoon. You should also be able to uninstall modules prior to D9 upgrade.

PS; emailed you a GovCMS slack invite, if you don't hear back from the GovCMS team (long weekend, so likely low staff/busy).

govCMS / GovCMS