search

clarity-h2020 / docker-drupal

Drupal 8 Project for implememting the CLARTIY CSIS Platform.
https://csis.myclimateservice.eu
GNU Lesser General Public License v3.0
3 stars 0 forks source link

Drupal update: fix security issue #53

Closed maesbri closed 5 years ago

maesbri commented 5 years ago

There is this recurring security issue message whenever I enter in drupal. Can Drupal components be updated to a newer version to fix this?

image

p-a-s-c-a-l commented 5 years ago

I've update Drupal core and most modules via drush but the following modules:

docker@3b8d70db32d6:/app$ composer update drupal/admin_toolbar > DrupalProject\composer\ScriptHandler::checkComposerVersion Loading composer repositories with package information Updating dependencies (including require-dev) Nothing to install or update Package phpunit/phpunit-mock-objects is abandoned, you should avoid using it. No replacement was suggested. Generating autoload files

However these modules could be updated via the UI. @patrickkaleta @fgeyer16 please advise.

fgeyer16 commented 5 years ago

How did you update the core via drush? As far as I know this command was removed from drush for Drupal 8.

composer outdated and admin/reports/updates in the Drupal UI are out of sync. for composer admin_toolbar is already in the current version. But JSON API is not the current version for composer. To have composer on the current state of versions always composer shouöd be used because it keeps track of versions itself and does not check the real versions, which are installed. To get JSON API and admin_toolbar in sync I suggest to do composer remove followed by composer require. In this way the current version should get installed.

For devel module: the version step is from major version 1 to major version 2 Composer does not update such version changes automatically. Change the version of devel in composer.json manually and run composer update afterwards.

p-a-s-c-a-l commented 5 years ago

Sorry, i used composer, Not drush! I‘m always confusing These two Tools 🧐

Von meinem iPhone gesendet

Am 01.03.2019 um 19:47 schrieb fgeyer16 notifications@github.com:

How did you update the core via drush? As far as I know this command was removed from drush for Drupal 8.

composer outdated and admin/reports/updates in the Drupal UI are out of sync. for composer admin_toolbar is already in the current version. But JSON API is not the current version for composer. To have composer on the current state of versions always composer shouöd be used because it keeps track of versions itself and does not check the real versions, which are installed. To get JSON API and admin_toolbar in sync I suggest to do composer remove followed by composer require. In this way the current version should get installed.

For devel module: the version step is from major version 1 to major version 2 Composer does not update such version changes automatically. Change the version of devel in composer.json manually and run composer update afterwards.

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub, or mute the thread.

p-a-s-c-a-l commented 5 years ago

Admin Toolbar 1.26 is in composer.js drupal/admin_toolbar": "^1.26", But Drupal still complains:

grafik

Installing DEVEL 2.0 bricks the site:

Uncaught PHP Exception Symfony\\Component\\Routing\\Exception\\RouteNotFoundException: "Route "devel.execute_php" does not exist." at /app/web/core/lib/Drupal/Core/Routing/RouteProvider.php line 201

p-a-s-c-a-l commented 5 years ago

Another update problem:

Update #8102

Failed: Drupal\Core\Database\DatabaseExceptionWrapper: SQLSTATE[42601]: 
Syntax error: 7 ERROR: syntax error at or near "NULL" LINE 1: ALTER TABLE br_schedule ADD COLUMN 
"update_entity" NULL ^: ALTER TABLE {br_schedule} ADD COLUMN "update_entity" NULL; Array ( ) in 
business_rules_update_8102() (line 40 of /app/web/modules/contrib/business_rules/business_rules.install).
p-a-s-c-a-l commented 5 years ago

More details:

docker@3b8d70db32d6:/app$ drush updb
 ---------------- ----------- --------------- ---------------------------------
  Module           Update ID   Type            Description
 ---------------- ----------- --------------- ---------------------------------
  business_rules   8102        hook_update_n   Allows schedule to work with
                                               event variables.
                                               https:www.drupal.orgprojectbusi
                                               ness_rulesissues3040833
 ---------------- ----------- --------------- ---------------------------------

 Do you wish to run the specified pending updates? (yes/no) [yes]:
 > yes

>  [notice] Update started: business_rules_update_8102
>  [error]  SQLSTATE[42601]: Syntax error: 7 ERROR:  syntax error at or near "NULL"
> LINE 1: ALTER TABLE br_schedule ADD COLUMN "update_entity"  NULL
>                                                             ^: ALTER TABLE {br_schedule} ADD COLUMN "update_entity"  NULL; Array
> (
> )
>
>  [error]  Update failed: business_rules_update_8102
 [error]  Update aborted by: business_rules_update_8102
 [error]  Finished performing updates.
fgeyer16 commented 5 years ago

This is a bug for which a patch exists: https://www.drupal.org/project/business_rules/issues/3043734#comment-13042740

p-a-s-c-a-l commented 5 years ago

This will be fixed in drupal/business_rules beta-8. I don't want to install a dev version or apply a patch manually.

p-a-s-c-a-l commented 5 years ago

@fgeyer16 Can you please fix this SSL issue with your repository:

The "https://repository.scienceatwork.eu/drupal/packages.json" file could not be downloaded: SSL operation failed with code 1. OpenSSL Error messages: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed Failed to enable crypto failed to open stream: operation failed https://repository.scienceatwork.eu/drupal could not be fully loaded, package information was loaded from the local cache and may be out of date

p-a-s-c-a-l commented 5 years ago

drupal/business_rules:1.x-dev cannot be installed as suggested here

``Your requirements could not be resolved to an installable set of packages. Problem 1

p-a-s-c-a-l commented 5 years ago

3043734-Update-to-beta7-fails-5.patch applied

p-a-s-c-a-l commented 5 years ago

@DenoBeno Cannot update from 8.6.17 to 8.7.5 due to the following error:

Updating dependencies (including require-dev)
Your requirements could not be resolved to an installable set of packages.

  Problem 1
    - webflo/drupal-core-require-dev 8.6.17 requires drupal/core 8.6.17 -> satisfiable by drupal/core[8.6.17] but these conflict with your requirements or minimum-stability.
    - webflo/drupal-core-require-dev 8.6.17 requires drupal/core 8.6.17 -> satisfiable by drupal/core[8.6.17] but these conflict with your requirements or minimum-stability.
    - webflo/drupal-core-require-dev 8.6.17 requires drupal/core 8.6.17 -> satisfiable by drupal/core[8.6.17] but these conflict with your requirements or minimum-stability.
    - Installation request for webflo/drupal-core-require-dev (locked at 8.6.17) -> satisfiable by webflo/drupal-core-require-dev[8.6.17].

Probably caused by this dependency: "webflo/drupal-finder": "^1.0.0".

Additionally, the following modules cannot be safely upgraded to the lasted version as they may break the current system:

grafik

patrickkaleta commented 5 years ago

There seem to be quite some difficulties when trying to upgrade from 8.6.x to 8.7.x. In my local instance, I tried upgrading Core using composer update drupal/core webflo/drupal-core-require-dev --with-dependencies as explained here (I tried this even though this command is mentioned specifically for upgrading from 8.5 to 8.6, since it was suggested in a couple of related issue reports).

In addition to what @p-a-s-c-a-l already encoutered I noticed that we will probably have to remove the JSON:API module, since it is now part of the Drupal Core (Drupal 8.7.0 release notes mention this) and it otherwise makes an upgrade impossible. My second problem occured when updating the database after the composer update with drush updatedb. I ran into this problem, which ultimately broke my local instance of the CSIS. Here might be a workaround for this problem, but I wasn't able to test it yet.

p-a-s-c-a-l commented 5 years ago

OK, then we'll defer the upgrade till the problems have been resolved locally. And we have to make sure that the update of the JSON:API doesn't break our external components.

patrickkaleta commented 5 years ago

And we have to make sure that the update of the JSON:API doesn't break our external components.

As far as I understood it, they simply moved the JSON:API version 2.4 into Drupal core, which is also the version of the module we are currently using. But you're right - better be safe than sorry.

fgeyer16 commented 5 years ago

It seems, that I was able to update my local insatnce succesfully. The steps Remove outdated:

composer remove drupal/jsonapi_extras
composer remove drupal/jsonapi
composer remove drupal/eva

~composer remove drupal/field_group~

update composer.json

"drupal/core":"~8.7.0",
"webflo/drupal-core-require-dev":"~8.7.0",

update drupal composer update drupal/core webflo/drupal-core-require-dev --with-dependencies reinstall modules

composer require drupal/eva
composer require drupal/jsonapi_extras

~composer require drupal/field_group~ run database update

vendor/bin/drush updb
vendor/bin/drush cr

found out eva and field_group were in the old version again so updated composer.json

"drupal/auto_entitylabel": "^3.0",
"drupal/eva":"^2.0",

~"drupal/field_group":"^3.0",~ update

composer update
vendor/bin/drush updb

drush command for updating entities was disabled so install a moduel and update entities

composer require drupal/devel_entity_updates
vendor/bin/drush en devel_entity_updates
vendor/bin/drush dentup
vendor/bin/drush cr

Map component seems to work Scenario Analysis seems to work the components on the table tabs show same behavior than on the live page. (Hazard Characterization - Local Effects shows something, the other not)

please check if procedure is reproduceable on your local machines

fgeyer16 commented 5 years ago

Sorry hit wrong button

fgeyer16 commented 5 years ago

Ok Forgot one point. We have to patch business rules after the update to drupal 8.7.x Issue: https://www.drupal.org/project/business_rules/issues/3016888 patch: https://www.drupal.org/files/issues/2019-05-10/variables-returning-null-3016888-24.patch

fgeyer16 commented 5 years ago

Group module needs to be re patched as well file web/modules/contrib/group/src/Entity/Group.php add the default form in the annotations 

     *     "form" = {
 +   *       "default" = "Drupal\group\Entity\Form\GroupForm",
     *       "add" = "Drupal\group\Entity\Form\GroupForm",
     *       "edit" = "Drupal\group\Entity\Form\GroupForm",
     *       "delete" = "Drupal\group\Entity\Form\GroupDeleteForm",
     *     },
patrickkaleta commented 5 years ago

@fgeyer16 thanks for the instructions. I will test this on my local machine and get back to you.

DenoBeno commented 5 years ago

We need install/update instructions in a wiki, or as a "best practice" item. Just a description of all the modules that need patching, special configuration options and such.

On Mon, 5 Aug 2019, 09:47 patrickkaleta, notifications@github.com wrote:

@fgeyer16 https://github.com/fgeyer16 thanks for the instructions. I will test this on my local machine and get back to you.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/clarity-h2020/docker-drupal/issues/53?email_source=notifications&email_token=AAWTC7UCBSLVPJETLV2O2TLQC7LJRA5CNFSM4G246DU2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3Q7NCY#issuecomment-518125195, or mute the thread https://github.com/notifications/unsubscribe-auth/AAWTC7SDEX2HE53S377DYKTQC7LJRANCNFSM4G246DUQ .

fgeyer16 commented 5 years ago

Concerning the modules, which need patching I suggest to do this using composer. In another projects my proceeding is the following:

  1. create a patches folder in the directory with the composer.json.
  2. Add the patch section in the extra section of the composer.json file:
{
  "name": "drupal-composer/drupal-project",
  "description": "Project template for Drupal 8 projects with composer",
.....
  "extra": {
    "patches": {
      "drupal/first_module_to_patch": {
        "patch description": "patches/patchfile.patch"     
      },
      "drupal/second_module_to_patch": {
        "patch description": "patches/patchfile.patchg"     
      },
    }
    "composer-exit-on-patch-failure": true,
    "patchLevel": {
      "drupal/core": "-p2"
    },
.....
}
  1. If there is a patch needed copy the patch file into this folder. We could also use the url for the patch path (e.g. https://www.drupal.org/files/issues/2019-05-10/variables-returning-null-3016888-24.patch). But I experienced two problems with that:

a. The url of the patch is not reachable (temporarily). So you have to remember to run composer update some when again to get the patch b. The path to the file to patch in the patch file is not correct due to different settings of diff when creating the patch.

In this way no needed patch can be forgotten during updates. If the targeted file of a patch has changed to much so patch cannot find the lines to change you will get an error message and can check if the patch is still needed, or make new pach which can be applied

patrickkaleta commented 5 years ago

Thanks @fgeyer16 , I tested your updating approach and it worked on my local instance. I did some slight changes, which I edited into your original approach. That way all modules (except the Field Group module) are up-to-date (as can then be checked here).

1) I wouldn't touch the Field_group module, since for version 3.0 it needs additional patches (mentioned here) when used on sites with the Paragraph module (which is the case for us.) I think we should take care of this module after the overall upgrade of the system and we should use the approach you mentioned above.

2) I changed the line in the composer.json regarding the auto_entitylabel module, so that it would be updated to the newest version as well.

3) Regarding updating the Admin Toolbar from 1.24 to 1.27 I noticed something strange. Checking the /web/modules/contrib folder I found a admin_toolbar and a admin_toolbar_compare folder. The admin_toolbar_compare folder was version 1.24 and used/shown by Drupal, while admin_toolbar was version 1.27. Simply removing the admin_toolbar_compare folder did the trick and Drupal was showing that it was now using the newest (1.27) version of this module. I don't know if this is the right approach and how this admin_toolbar_compare folder even got there in the first place.

@fgeyer16 If you don't oppose the changes I made to your original approach, I'd like to apply those changes on the server now (ideally together with you via Telco just as a precaution)

patrickkaleta commented 5 years ago

Concerning the modules, which need patching I suggest to do this using composer. In another projects my proceeding is the following: ...

I like this idea and I would even go a step further and exclude the patches into a separate composer.patches.json file (place it in same directory as the composer.json) in order to prevent the composer.json from becoming too overcrowded. For the stored patch files on our server I would suggest the following naming convention:[Module affected]_[Issue number on Drupal.org].patch

composer.json file would then look something like this:

{
  "name": "drupal-composer/drupal-project",
  "description": "Project template for Drupal 8 projects with composer",
.....
  "extra": {
    "patches-file": "composer.patches.json",
    "enable-patching": true,
    "composer-exit-on-patch-failure": true,
    "patchLevel": {
      "drupal/core": "-p2"
    },
.....
}

and the composer.patches.json would look for example like this:

{
  "patches": {
    "drupal/core": {
      "Issue #2920527": "patches/core_2920527.patch"
    },
    "drupal/group": {
      "Issue #2833054": "patches/group_2833054.patch"
    },
    "drupal/business_rules": {
      "Issue #2994155": "patches/business_rules_2994155.patch",
      "Issue #3016888": "patches/business_rules_3016888.patch"
    }
  }
}

@fgeyer16 let me know what you think about this and also when you would be available, so that we can get those updates & patches done.

patrickkaleta commented 5 years ago

Drupal and all modules (except "Field Group" and "Admin Toolbar" - will be done separately) have been updated. All should be working as before, but please check for yourself.

p-a-s-c-a-l commented 5 years ago

I couldn't find composer.patches.json in /docker/100-csis/drupal-data.

patrickkaleta commented 5 years ago

I couldn't find composer.patches.json in /docker/100-csis/drupal-data.

I have yet to download and move the existing patches into the dedicated composer.patches.json. I will do this together with updating the remaining two modules.

patrickkaleta commented 5 years ago

Remaining modules have been updated and all patches have been moved to the composer.patches.json in /docker/100-csis/drupal-data. I will create a new "Best practice" regarding the patching of modules. Closing this issue now.