Allow spx_profiler_start + spx_profiler_end to be used on non-CLI

[dochne]

Using the spx_profiler_start + spx_profiler_end functions it's possible to pull a task from a queue, run it, then save the profiled result elsewhere so you can view profiles of particularly slow tasks.

When looking into slow tasks, it can be useful to run the task processing inside a web UI. Ideally these should still be able to give you a profile.

As it stands, it is not currently possible to run these functions inside a non-CLI environment.

[NoiseByNorthwest]

When looking into slow tasks, it can be useful to run the task processing inside a web UI. Ideally these should still be able to give you a profile.

As I have said previously, it should be easy to implement (if it is just a guard to remove).

But I'm a bit curious about your use case. If I have understood correctly, for profiling purpose, you run through an HTTP handler SAPI (fpm, apache...) several tasks within the same script execution like a CLI queue consumer ? Is that right ? If so, why is it simpler that simply running one task at a time via a CLI script so that you just have to profile the whole script ? After all if you can run these tasks via a queue consumer you may be able to directly run them within a regular CLI script ?

[dochne]

It's largely about convenience. I'm trying to get people to care a bit more about the performance of their code and the best way to do that (from my perspective) is to try and make the profiling process as easy as possible.

In our dev environment we now profile everything, then, if the execution took over a certain amount of time, we store the profile against that task, upload it to s3 and log a reference. This allows us to have excellent visibility over slow tasks and start digging into the problem very quickly.

I would like to be able to add functionality that means that people can easily rerun the task in a friendly UI and get an up to date profile.

This is particularly of use in two usecases:

1. local dev environments where PHP devs are iteratively changing things to try and fix performance bottlenecks and
2. production, where we don't want to be running spx on tasks as a matter of course, but would love the ability to be able to get profiling data on an ad-hoc basis

Hopefully this makes sense!

[NoiseByNorthwest]

In our dev environment we now profile everything

I hope you use a sampling period > 0, do you ? Otherwise it would be an important waste of resources to proactively profile everything in tracing mode. Of course with sampling mode you'll get statistically accurate reports but it is also often more accurate than the tracing mode for time related resources.

You could also log & monitor the resource consumption at script or task level, no profiler needed for this, just few lines of PHP (using microtime(true)). And then profile only the scripts which appear to be too greedy regarding your requirements.

I would like to be able to add functionality that means that people can easily rerun the task in a friendly UI and get an up to date profile.

How this UI works ? Does it send the order to execute the task to the app via an HTTP request (XHR..) which synchronously executes the task as I imagine it ? If so this is a single task per script model. Why do you need to start / stop ? Is it because you want the profiling report to focus on the executed task and you don't want it to get polluted by the outer parts (before & after) ?

[dochne]

I hope you use a sampling period > 0, do you ?

(To clarify, this is our pre-staging environment)

Yup, it'd be super painful (and pretty inaccurate) otherwise! I am pretty aware of the performance overheads involved, but thanks for the warnings!

How this UI works ? Does it send the order to execute the task to the app via an HTTP request (XHR..) which synchronously executes the task as I imagine it ?

Pretty much. Standard HTTP POST request rather than Ajax, but same thing really.

If so this is a single task per script model. Why do you need to start / stop ? Is it because you want the profiling report to focus on the executed task and you don't want it to get polluted by the outer parts (before & after) ?

The lack of pollution is nice, but it's really about being able to programmatically interact with the result. The task execution code looks something like this (psuedocode), and is shared between the CLI + UI logic.

class TaskProcessor
{
  public function __constructor(private TaskExecutor $this->taskExecutor, private ProfileStorage $this->profileStorage)
  {}

  public function run(Task $task, bool $forceProfile = false) {
    $startTime = microtime(true);
    spx_profile_start();
    $this->taskExecutor->execute($task);
    $key = spx_profile_end();
    $duration = microtime(true) - $startTime;
    if ($forceProfile || $duration > 10) {
      $this->profileStorage->save($task, $key);
    }
  }
}

While you can happily profile the script as a whole by setting the cookie and reloading the page (which is how I do it currently), you lose the ability to interact with the profile results on your terms as part of the script.

[NoiseByNorthwest]

OK I got it, the instrumentation is in the task processor (same code on CLI/consumer & HTTP sides), and furthermore you need the report key. It makes sense.

[NoiseByNorthwest]

I will add:

• the spx.http_profiling_auto_start INI parameter
• the support of SPX_AUTO_START for non-CLI
• the support of SPX_AUTO_START in web UI / control panel