Integrity check

[danielbachhuber]

When performing a core, theme or plugin update, it would be helpful if WP-CLI ran an integrity check before and after the update, so it was easier to know whether the update created any problems.

What sort of heuristics would WP-CLI check?

[TheLastCicada]

A comparison of the homepage output before and after the update would be nice to have. On some sites, this output is going to vary from page load to page load if there's highly dynamic data, but on most sites I work on, very little should be different on the homepage before and after a core or plugin update. Would be great to be able to get an alert if there's a difference in output before and after the update and a way to view what the difference is.

Along the lines of your profile command, having a check of the page load time before and after an update would be good too. Would want to make sure the cache purge at update doesn't skew these results though. Could display a warning if there's a 20%+ difference in page load performance or something like that.

[anantshri]

some times i have seen plugin's adding a caption on homepage which you only realize exist's when you login via GUI. IT might be a good idea if we can show that after a plugin install a banner is added. example recently stop-user-enumeration added settings and a banner was all the time present till someone clicked on save settings in options.

[danielbachhuber]

From some research I did:

• Automattic runs mShots, a tool that can produce a visual screenshot: https://github.com/Automattic/mShots
• ImageMagick has some algorithm to compare two image files: http://www.imagemagick.org/Usage/compare/

[danielbachhuber]

Also, from BlueHost's wp-tools:

• wp eval 'echo "ok";' executes successfully
• Compare content-length and status code of homepage and wp-admin

[kirtangajjar]

Compare content-length and status code of homepage.

Checking status code would be good idea to test if everything works correctly, but I'm not sure how content-length could help.

[danielbachhuber]

I've actually written an initial implementation of this: https://github.com/danielbachhuber/update-verify

[swissspidy]

Is this perhaps already covered by the automatic rollback feature in core itself?

[danielbachhuber]

Core's checks are relatively minimal. They seem to focus exclusively on the filesystem operations:

https://github.com/WordPress/wordpress-develop/blob/4ff71a6adf6f20966938331f7eee20b98ad4ed4c/src/wp-admin/includes/update-core.php#L1057-L1610

The update verify package I wrote for DreamHost checks these heuristics:

• HTTP status code (200 or otherwise).
• Whether or not a PHP fatal error is detected.
• Whether or not the closing tag is detected.

If I recall correctly, it was pretty successful for updating thousands of out of date WordPress installs.

I don't know that there's anything super actionable and specific in this thread, so we can close it.