Persistent WP instance to speed up WP-CLI

[kkmuffme]

Feature Request

• [x] Yes, I reviewed the contribution guidelines.

Describe your use case and the problem you are facing

Since WP CLI always initiates a new PHP CLI/WP process, you have the WP overhead for every command, which makes it the slowest part of many pipelines. Additionally having to specify common commands every time is a chore to write, e.g. --skip-plugins --skip-themes to make it faster.

if wp plugin is-active foo
then
    echo "do something"
fi

if wp plugin is-active bar
then
    echo "do something else"
fi

wp option update "hello" "world"

Describe the solution you'd like

Since PHP CLI has no/long timeouts, it would make sense to give the option to "launch" a WP instance once and then reuse it.

It would be faster when we launch a WP instance and then just read a file in a while loop like https://serverfault.com/a/1088115 to execute the commands passed subsequently. This could be an optional thing with --persist=my-instance for example.

Since the WP overhead is between 150-350ms on a barebones WP install and much higher on slow servers or with many plugins, this would save tons of runtime.

[schlessera]

The idea is generally interesting. However, we need to be aware of some limitations regarding such an approach. These mostly come from the fact that this goes against the design principles and architecture of WordPress itself. WordPress is built in such a way that each request is meant to start a new process from scratch. This means that there are no precautions being taken within the active WP process to deal with multiple changes that depend on each other's side-effects. As an example, depending on what you're trying to do, you'd need to manually invalidate/reset some caches between one WP-CLI call and the next to make sure that the second one actually sees the changes the first one did.

How would you consider solving against this? Would it be okay to put the onus on the user and expect them to do a wp cache flush ... in-between other calls where needed? What about the stuff that gets put into constants and still can be changed by WP-CLI calls?

[toddr]

How would you consider solving against this? Would it be okay to put the onus on the user and expect them to do a wp cache flush ... in-between other calls where needed? What about the stuff that gets put into constants and still can be changed by WP-CLI calls?

The constants make things complicated... Is there any way to pre-compile the code prior to capturing system state, then fork for each command? My idea breaks of course if you do something like install a plugin or upgrade wordpress, etc... Possibly the orchestrator would somehow know to "reload" when that happens.

I realize this is all a stretch. But maybe it will lend some ideas.

[toddr]

A lesser proposal to this idea might be to do something like this. Would it solve the problem you're trying to solve @kkmuffme?

echo 'set options blogname "my title"' > commands.txt
echo 'set options blogdescription "my description"' >> commands.txt
echo 'core update' >> commands.txt
wp-cli --json runcommands << commands.txt

Possibly this command would come with a caveat emptor that it may have unexpected side effects. In my above example, I would be required to know that messing with blogname/description will not have side effects on each other and that the update would be fine following the first 2. If we wanted to get fancy, maybe we'd add a flush option that would invalidate caches if needed as you go?

The JSON result might provide an array of responses from each command.

[kkmuffme]

@schlessera

Would it be okay to put the onus on the user and expect them to do a wp cache flush ... in-between other calls where needed?

Yes. As this goes far beyond the use case of an average WordPress user. This is mostly for commands that would run within the same WP request normally too - e.g. updating multiple options,...

For most use cases, there's no special requirement - and in fact in some cases, we can automatically end the instance and create a new one internally - e.g. on plugin activation, to fix this issue automatically.

---

@toddr unfortunately I can't do this, since all the "if" code won't work then - this works fine for updating/deleting options, but won't work when I have to check if a plugin is active,... making it too restrictive.

---

The simplest version of it would be to start a WP request and put a while/sleep after some specific hook. And on this hook it then checks if any new commands were added to stdin/pipe and executes them.

while( $line = fgets( ... ) ) {
//$line is whatever task, e.g. update option,.. and gets executed now.
if ( $line === 'exit' ) {
break; // end the persistent instance
}
usleep( 100 );
}

Basically making a WP request infinite (or until CLI timeout is reached or it's closed).

[toddr]

As this goes far beyond the use case of an average WordPress user. This is mostly for commands that would run within the same WP request normally too - e.g. updating multiple options,...

It almost sounds like you want to create your own CLI plugin that could then do complex things that you need and could handle the conditionals?

[toddr]

Unfortunately I can't do this, since all the "if" code won't work then - this works fine for updating/deleting options, but won't work when I have to check if a plugin is active,... making it too restrictive.

Cool. I've moved my idea to #180

[kkmuffme]

It almost sounds like you want to create your own CLI plugin that could then do complex things that you need and could handle the conditionals?

Not really, since this is a general issue with WP CLI - unless you use WP CLI to update a single option or do a single operation (which is rather unlikely) you will have this issue

[schlessera]

I realized another thing that needs consideration here: If we have an active WordPress process listening to outside commands, we open up a whole can of security implication worms. As an example, you might have needed to run the WP-CLI process under a different user to make it work with whatever permission restrictions are in place. But then, WP-CLI would need to be aware of this to know what user to accept additional input from and what to refuse. So you'd need some form of authentication and/or authorization mechanism, as you're circumventing the shell and OS restrictions.

[kkmuffme]

Depending on how this is implemented, the security is already handled by default Linux file permissions. e.g. the linked serverfault solution using the pipe, will be created with default umask permissions of the current user/directory.

[nickchomey]

Perhaps I missed it, but I haven't seen anyone address @schlessera's comments about how this goes completely against the architecture of Wordpress.

A few months ago I spent the better part of a week exploring how to run Wordpress itself (not wpcli) in one of the various async php servers (swoole, roadrunner, reactphp, frankenphp, etc...) and the conclusion is fundamentally that it would require a complete reworking of Wordpress Core and probably most plugins as, for example, they all create and use global variables that would change/interfere with each request.

So, if something were ever to be attempted here, it would be better/necessary to instead do it for WP Core itself. I don't see that ever happening...

[kkmuffme]

That's not an issue for most WP CLI commands since all the globals used in WP are for states of a specific page only, but WP CLI is stateless in general, just like e.g. wp-cron.php is. If you could provide an example where/which global variable you think would be a problem for this, it'd be appreciated though, in case I missed something.

[nickchomey]

Ah ok, I guess I need more experience with wp cli! I only stumbled upon this while starting my research into making a package for it.