Automattic / jetpack

Security, performance, marketing, and design tools — Jetpack is made by WordPress experts to make WP sites safer and faster, and help you grow your traffic.
https://jetpack.com/
Other
1.59k stars 798 forks source link

Update to 11.8 breaks WAF again, requiring manual intervention. #28819

Closed Ipstenu closed 1 year ago

Ipstenu commented 1 year ago

Impacted plugin

Protect, None / Other

Quick summary

After upgrading to 11.8, my site bombed with a critical error.

Error Details
=============
An error of type E_ERROR was caused in line 114 of the file /home/USER/[lezwatchtv.com/wp-content/plugins/jetpack/jetpack_vendor/automattic/jetpack-waf/src/class-waf-initializer.php](http://lezwatchtv.com/wp-content/plugins/jetpack/jetpack_vendor/automattic/jetpack-waf/src/class-waf-initializer.php). Error message: Uncaught Error: Call to undefined method Automattic\Jetpack\Waf\Waf_Constants::define_mode() in /home/USER/[lezwatchtv.com/wp-content/plugins/jetpack/jetpack_vendor/automattic/jetpack-waf/src/class-waf-initializer.php:114](http://lezwatchtv.com/wp-content/plugins/jetpack/jetpack_vendor/automattic/jetpack-waf/src/class-waf-initializer.php:114)
Stack trace:
#0 /home/USER/[lezwatchtv.com/wp-includes/class-wp-hook.php](http://lezwatchtv.com/wp-includes/class-wp-hook.php)(308): Automattic\Jetpack\Waf\Waf_Initializer::check_for_waf_update('')
#1 /home/USER/[lezwatchtv.com/wp-includes/class-wp-hook.php](http://lezwatchtv.com/wp-includes/class-wp-hook.php)(332): WP_Hook->apply_filters(NULL, Array)
#2 /home/USER/[lezwatchtv.com/wp-includes/plugin.php](http://lezwatchtv.com/wp-includes/plugin.php)(517): WP_Hook->do_action(Array)
#3 /home/USER/[lezwatchtv.com/wp-admin/admin.php](http://lezwatchtv.com/wp-admin/admin.php)(175): do_action('admin_init')
#4 /home/USER/[lezwatchtv.com/wp-admin/plugins.php](http://lezwatchtv.com/wp-admin/plugins.php)(10): require_once('/home/USER...')
#5 {main}
 thrown

In order to fix this I did:

  1. deactivate WAF in wp-cli
  2. Remove call to WAF from php.ini
  3. Edit jetpack/jetpack_vendor/automattic/jetpack-waf/src/class-waf-initializer.php so function public static function check_for_waf_update() { immediately returns without anything.

At this point, I was able to log in. As soon as I could, I removed my edit in item 3, refreshed, and was fine, so I reactivated the module and then restored the line to php.ini

Steps to reproduce

  1. Ensure WAF is active
  2. update plugin

A clear and concise description of what you expected to happen.

Nothing should break

What actually happened

Critical fail emails :(

Impact

One

Available workarounds?

Yes, difficult to implement

Platform (Simple and/or Atomic)

No response

Logs or notes

No response

samiff commented 1 year ago

Thanks for the report @Ipstenu

I'm having some trouble reproducing this on my end. I've tried updating from Jetpack v11.7.1 to v11.8 with the WAF module enabled, but wasn't able to see the same error trigger.

Two questions:

  1. Which version of Jetpack were you upgrading from?
  2. Could you tell me a bit more about the step you mentioned: "Remove call to WAF from php.ini"

Edit: internal discussion p1675805859785799-slack-C029WFNV69M

Ipstenu commented 1 year ago
  1. 11.7.1 - I'm always on latest as I auto update
  2. I use auto_prepend

To ensure the firewall can best protect your site, please update: auto_prepend_file PHP directive to point to /home/USER/lezwatchtv.com/wp-content/jetpack-waf/bootstrap.php Typically this is set either in an .htaccess file or in the global PHP configuration; contact your host for further assistance.

In my case it goes in .user.ini, but since more people know it as php.ini, I called it that. It's the custom PHP file that lets you make tweaks.

I know how sensitive that is, and had it only been "Remove that line, refresh, save, restore the line" I don't think it's really a bug, but even after I did that, I STILL had to go in and kill that function tells me something's not happy.

My initial thought was "Oh it's running the update WAF too early"

jeherve commented 1 year ago

Thanks for the extra details!

@nateweller Do you think you could take a look at this?

aleone89 commented 1 year ago

There's been a report of this via Twitter: https://twitter.com/saqksahm/status/1623336373225951232

nateweller commented 1 year ago

Thank you for the report and all of the details @Ipstenu! 🙇‍♂️

We've just merged #28844, which will prevent this from happening in the future.

There's been a report of this via Twitter: https://twitter.com/saqksahm/status/1623336373225951232

This tweet has a different stack trace (fails attempting to update the firewall rule files), but the above PR will prevent that specific case from occurring as well. Additionally, we've also merged #28760 to prevent these types of fatals going forward when saving rule files.

Ipstenu commented 1 year ago

@nateweller That brings up a new question based on #28401

Should the directions tell me to link my prepended file to jetpack-protect (which I legit had not realized was a separate plugin -- I don't look at that as often as all that ;) )?

nateweller commented 1 year ago

Should the directions tell me to link my prepended file to jetpack-protect (which I legit had not realized was a separate plugin -- I don't look at that as often as all that ;) )?

@Ipstenu The prepended wp-content/jetpack-waf/bootstrap.php file will (now) automatically point to the latest available version of the firewall code, so there should be no extra steps required!

Whether you are using the Jetpack plugin, the Jetpack Protect plugin, or both at the same time, the auto_prepend_file value should always be that single bootstrap.php file in wp-content, which will load the appropriate firewall code to run from whichever plugin has the latest and greatest firewall code.

Ipstenu commented 1 year ago

Awesome! Thank you, @nateweller - 5 stars :D