Add `--force` flag to `wp plugin list` and `wp theme list`

[baizmandesign]

Hi! Here's a proposal to add a new option to wp plugin list and wp theme list: --force.

Why am I proposing this option? Because when I list plugins or themes, WP CLI first checks the "_site_transient_update_{type}s" transient in the options database. (This transient seems to get updated on an occasional basis, but I'm uncertain of its refresh interval.) This often leads to misleading information when checking for available plugin and theme updates. The idea behind --force would delete the transient, forcing the system to fetch the latest version information before displaying it.

Example usage:

$ wp plugin list --force

The name --force may not be the most appropriate choice as it doesn't fully explain what the command is doing and why it's needed. Maybe --force-update or --refresh might work better?

If this gets approved, I could take a crack at authoring it.

[danielbachhuber]

Thanks for the suggestion, @baizmandesign !

Both wp plugin list and wp theme list do a forceful refresh by default when listing their items:

https://github.com/wp-cli/extension-command/blob/955bd947d95ef91da501179c7e561d229ab3cc19/src/WP_CLI/CommandWithUpgrade.php#L497-L500

Maybe that's not working on your system for some reason?

[baizmandesign]

@danielbachhuber Thanks for your reply!

After reading the code at the link you provided, it appears to call wp_update_plugins() or wp_update_themes(). Both of these functions live in wp-includes/update.php of WordPress. To take plugins as an example, a transient is set on line 393 of wp-includes/update.php:

set_site_transient( 'update_plugins', $current );

Notably, there's no argument for an expiration (which would have been the third parameter). This would seem to indicate there is no expiration, but in practice, in my experience, the transient gets refreshed every so often (I'm uncertain how often).

The way that I've even been able to observe this behavior is because I'm running my own WordPress plugin and theme update server. It's possible that my code has an error in it. The only way to reproduce this issue, and confirm this is indeed occurring, is to have your own update server. Or perhaps, if one had a plugin or theme on the WordPress repository, to 1) list your plugins (or themes), 2) update the asset on the WordPress repository, and 3) the list your plugins (or themes) again. If I'm right, WordPress won't immediately see the newly available version.

At this point, I feel as if I'm claiming there's a problem and offering a solution, and you're saying there's no problem. We don't have a good way to reproduce the "issue" I've raised. How do we proceed?

[danielbachhuber]

Oh, I see. I assumed wp_update_plugins() always made the outbound HTTP request. It looks like it bails early if there was a "recent" check: https://github.com/WordPress/wordpress-develop/blob/44a2073816ba8c474f598c348e4d334696b495e4/src/wp-includes/update.php#L385-L388

Today I learned!

wp core check-update has a --force-check flag, so let's add the same to wp plugin list and wp theme list. We can forcefully trigger the update check by deleting the update_plugins transient when the flag is present.

When writing tests, you can write a test value to the update_plugins transient that persists when the flag isn't present, and gets updated when the flag is present.

[baizmandesign]

@danielbachhuber I like the idea of adding a --force-check flag to the plugin list and theme list commands. (Part of me thinks this should be the default behavior and not require an extra flag, but I also understand why you would want WP CLI to mirror the behavior of the web interface.)

When this is implemented, can I simply configure my .wp-cli/config.yml to always pass this flag to the list commands? I checked the handbook webpage on configuring WP CLI, and I think I can like so:

plugin list:
  force-check:
theme list:
  force-check:

Will this work?

[danielbachhuber]

@baizmandesign Yep, it should work as you describe.

[baizmandesign]

@danielbachhuber Good news: I've updated the relevant code in the extension-command repository. It was trivial. (View the WIP here.)

Less good news: I'm uncertain how to proceed with the testing portion. (Yes, I read the relevant portion of the handbook.) When I run composer prepare-tests, the wp_cli_test database is created, but it's empty. Shouldn't it have typical WordPress tables in it? There were no error messages after running composer prepare-tests.

How do I test this new functionality against an empty database? Also, do I need to update the Behat files or write unit tests for PHPUnit or both? I'm a bit out of my depth here. Any guidance would be appreciated.

Here are the Behat files that I think need to be updated:

• extension-command/features/plugin.feature
• extension-command/features/theme.feature

(Note, the plugin list command appears in multiple *.feature files: plugin-auto-updates-disable.feature, plugin-auto-updates-enable.feature, plugin-list-wporg-status.feature, plugin-update.feature, and plugin.feature. I'm uncertain which file is the most appropriate home for this test.)

[danielbachhuber]

When I run composer prepare-tests, the wp_cli_test database is created, but it's empty. Shouldn't it have typical WordPress tables in it? There were no error messages after running composer prepare-tests.

@baizmandesign Not necessarily, no. It just sets up an empty database. The tables are created and destroyed for each test.

How do I test this new functionality against an empty database? Also, do I need to update the Behat files or write unit tests for PHPUnit or both? I'm a bit out of my depth here. Any guidance would be appreciated.

You'll want to update the Behat files. If the written tutorial doesn't quite click, want to watch this hack day video? https://github.com/wp-cli/wp-cli/issues/5858

Here are the Behat files that I think need to be updated:

plugin.feature and theme.feature would be fine. You could also create plugin-list.feature and theme-list.feature files too.

[baizmandesign]

@danielbachhuber Thanks for the link to the video. I watched it, and it was helpful. More helpful was reading the tests in some of the other *.feature files; they gave me a better idea of how to write new tests.

What I was really unclear about, and did not articulate, was the contents of the database and the presence of plugins and themes in the test WordPress installation. (That's why I was initially confused about the creation of an empty database.) After writing some tests that intentionally failed, I was able to see the output of the wp plugin list and wp theme list commands. The behat output also spat out the name of the temporary directory where the test version of WordPress was installed.

My next question is about your earlier suggestion:

...you can write a test value to the update_plugins transient that persists when the flag isn't present, and gets updated when the flag is present.

It turns out that one can't simply set the value of the update_plugins transient to any value. When I set it to "test_value" and run wp plugin list, "test_value" gets deleted and replaced with the correct (and valid) transient value (a serialized array). I think I have to set the transient to a legal (fake) value.

Here's the serialized array in the database:

O:8:"stdClass":4:{s:12:"last_checked";i:1707859831;s:8:"response";a:0:{}s:12:"translations";a:0:{}s:9:"no_update";a:2:{s:19:"akismet/akismet.php";O:8:"stdClass":10:{s:2:"id";s:21:"w.org/plugins/akismet";s:4:"slug";s:7:"akismet";s:6:"plugin";s:19:"akismet/akismet.php";s:11:"new_version";s:5:"5.3.1";s:3:"url";s:38:"https://wordpress.org/plugins/akismet/";s:7:"package";s:56:"https://downloads.wordpress.org/plugin/akismet.5.3.1.zip";s:5:"icons";a:2:{s:2:"2x";s:60:"https://ps.w.org/akismet/assets/icon-256x256.png?rev=2818463";s:2:"1x";s:60:"https://ps.w.org/akismet/assets/icon-128x128.png?rev=2818463";}s:7:"banners";a:2:{s:2:"2x";s:63:"https://ps.w.org/akismet/assets/banner-1544x500.png?rev=2900731";s:2:"1x";s:62:"https://ps.w.org/akismet/assets/banner-772x250.png?rev=2900731";}s:11:"banners_rtl";a:0:{}s:8:"requires";s:3:"5.8";}s:9:"hello.php";O:8:"stdClass":10:{s:2:"id";s:25:"w.org/plugins/hello-dolly";s:4:"slug";s:11:"hello-dolly";s:6:"plugin";s:9:"hello.php";s:11:"new_version";s:5:"1.7.2";s:3:"url";s:42:"https://wordpress.org/plugins/hello-dolly/";s:7:"package";s:60:"https://downloads.wordpress.org/plugin/hello-dolly.1.7.3.zip";s:5:"icons";a:2:{s:2:"2x";s:64:"https://ps.w.org/hello-dolly/assets/icon-256x256.jpg?rev=2052855";s:2:"1x";s:64:"https://ps.w.org/hello-dolly/assets/icon-128x128.jpg?rev=2052855";}s:7:"banners";a:2:{s:2:"2x";s:67:"https://ps.w.org/hello-dolly/assets/banner-1544x500.jpg?rev=2645582";s:2:"1x";s:66:"https://ps.w.org/hello-dolly/assets/banner-772x250.jpg?rev=2052855";}s:11:"banners_rtl";a:0:{}s:8:"requires";s:3:"4.6";}}}

How do I set a transient value to an array? I've been using wp transient, which works fine for strings, but do I need to consider using wp eval instead?

[danielbachhuber]

How do I set a transient value to an array? I've been using wp transient, which works fine for strings, but do I need to consider using wp eval instead?

@baizmandesign You can simply write the value directly to the database using a PHP script. Here's one such example:

https://github.com/wp-cli/entity-command/pull/460/files#diff-5f720a02874bbc43f5f6bd95734838a0a15a72ec5790126cc89120a7d1c4dc96R382-R389

[baizmandesign]

@danielbachhuber This is exactly what I needed. Thanks!

[baizmandesign]

@danielbachhuber I've attempted to create a test that writes a fake value to the database, and I've failed. At this point, I've spent several hours on this task and am ready to give up.

Here is the problem. In my Behat test, every time I call wp plugin list, the plugin whose version I've faked (hello-dolly) reports the correct version (1.7.2), not my fake version (10.0.0).

You can view my WIP. The file with the relevant tests is plugin-list.feature.

My first approach entailed getting the plugin transient value for a specific plugin, manually setting it to a fake version number, and then saving the transient back to the database. That's what's happening in setup.php.

My second approach took inspiration from plugin.feature. I tried to create a must-use plugin in wp-content/mu-plugins/test-plugin-update.php which added a filter hooked to site_transient_update_plugins that returned a fake version number. This just didn't work. Running the list command returned the correct plugin version and not my fake version.

(At the moment, some of the tests are written to fail so that I can examine the output more closely. For example, running wp plugin list and expecting STDOUT to be empty doesn't make sense.)

I don't like giving up, but without serious guidance, I'm done.

[danielbachhuber]

@baizmandesign I appreciate your effort!

Can you put your work into a pull request and then I'll see if I can figure it out?

[baizmandesign]

All set, @danielbachhuber! I'm sorry I wasn't able to complete creating the tests. If there's anything else I can do, please let me know. WP CLI is an incredibly valuable tool and very important to me!