Advise regarding Timer usage

[Draghmar]

I have small API serwer based on Swoole. Now I need to do periodically some stuff so I wanted to use Timer but I wonder what would be the best approach for my case. My API uses ConnectionPool to prepare some PDO connections. I will need PDO also in Timer but also a MongoDB connection. First of all: should I start Timer in completely different file? So it will be a separate process, with it's own service file?

And for the next: Do I need ConnectionPool for operations within tick?

// Option #1
$dbpool = new ConnectionPool(function() use ($config) {
    return new DB($config);
}, 10);
$db = $dbpool->get();
$mongo = new MongoDB\Driver\Manager($creds);
Swoole\Timer::tick(3000, function($tid) use ($db,$mongo) {
    // Do some tuff with PDO and MongoDB.
});

That seems weird in terms of PDO because I don't actually know where to use $dbpool->put($db)...Of course there's this option:

// Option #2
$dbpool = new ConnectionPool(function() use ($config) {
    return new DB($config);
}, 10);
$mongo = new MongoDB\Driver\Manager($creds);
Swoole\Timer::tick(3000, function($tid) use ($db,$mongo) {
    $db = $dbpool->get();
    // Do some tuff with PDO and MongoDB.
    $dbpool->put($db);
});

But maybe there's no need for pool here at all so it should simply look like this:

// Option #3
$db = new DB($config);
$mongo = new MongoDB\Driver\Manager($creds);
Swoole\Timer::tick(3000, function($tid) use ($db,$mongo) {
    // Do some tuff with PDO and MongoDB.
});

And what about situation where there's no need to launch Timer as a separate process so it could go with the API? As I see it, it could be like this:

$dbpool = new ConnectionPool(function() use ($config) {
    return new DB($config);
}, 10);

// Here should go one of the three options from above to get the Swoole\Timer::tick running

$srv = new Server('127.0.0.1', 9501);
$srv->set([
    'log_level' => 0,
    'log_file' => '/var/log/swoole/swoole.log',
    'log_rotation' => SWOOLE_LOG_ROTATION_DAILY,
    'reload_async' => true,
    'http_compression_level' => 5,
    'http_gzip_level' => 5,
    'http_parse_post' => true,
]);

$srv->on('request', function($request, $response) use ($srv, $config, $dbpool) {
    // Do some API related stuff
});

[deminy]

The official MongoDB driver is not coroutine-friendly; it works in blocking mode in some cases.

A straightforward solution is to have MongoDB queries run in task workers; each task worker maintains a persistent connection to MongoDB.

A better solution is to use framework Hyperf + component reasno/fastmongo. For periodic checks, use component hyperf/crontab.

[Draghmar]

Thanks for the tip. Yeah, I've seen in the docs that MongoDB doesn't play well with Swoole in terms of coroutines. But I don't mind that in my scenario - it will only do one thing to get data and then it won't be needed until next tick. I'd rather like to know how to deal with SQL in tick - which of the presented options should I use to get it properly working.

[deminy]

In this case, you can use a connection pool, although it's not that necessary, since only the first connection in the pool will be used. Connection pool is useful only when the DB drivers are coroutine-friendly (which means those drivers can make queries in non-blocking mode). Thus, option three is better than the first two.

Execution time of the callback function in Timer::tick() doesn't have effect on when the callback will be executed next time. e.g., if you set to run the callback to run every 3 seconds

Swoole\Timer::tick(3000, function() {
    // Run the query.
});

, and here is how to callback function (the query) will be executed:

00:00:00: Program starts
00:00:03: A query starts and it takes 2 seconds to finish
00:00:06: A query starts and it takes 4 seconds to finish
00:00:12: A query starts

As you can see there isn't a query started at 00:00:09 because the second query takes too much time to finish.

If you want the timer works differently, you will need to make some adjustments. For example, we can use the sleep function call instead:

while (true) {
  sleep(3); // Enable runtime hook so that we can make some async operations during this time.
  // Run the query.
}

[Draghmar]

Thanks for the tips! Nah, I actually like the fact that it won't fire up until previous task will be finished - suits my needs. :)

Now the only thing I need to know is how to start Timer? I tried that in the same file where HTTP server is started but I've got:

Warning: Swoole\Server::start(): eventLoop has already been created, unable to start Swoole\Http\Server

And now I'm not sure if it's because I should do that in a separate file or I'm doing something wrong. The simplified code looks like this:

$dbpool = new ConnectionPool(function() use ($config) {
    return new DB($config);
}, 10);

$db = $dbpool->get();
$mongo = new MongoDB\Driver\Manager($creds);
Swoole\Timer::tick(3000, function($tid) use ($db,$mongo) {
    // Do some tuff with PDO and MongoDB.
});

$srv = new Server('127.0.0.1', 9501);
$srv->on('request', function($request, $response) use ($srv, $config, $dbpool) {
    // Do some API related stuff
});
$srv->start();

[matyhtf]

The timer should be added to the callback function of the worker process start

$srv = new Server('127.0.0.1', 9501);
$srv->on('WorkerStart', function ($srv, $id) {
    $dbpool = new ConnectionPool(function() use ($config) {
        return new DB($config);
    }, 10);

    $db = $dbpool->get();
    $mongo = new MongoDB\Driver\Manager($creds);
    Swoole\Timer::tick(3000, function($tid) use ($db,$mongo) {
        // Do some tuff with PDO and MongoDB.
    });
});
$srv->on('request', function($request, $response) use ($srv, $config, $dbpool) {
    // Do some API related stuff
});
$srv->start();

[Draghmar]

@matyhtf Wouldn't it be better to set Timer at on('Start')? I mean, there can be more than one worker spawned, right? So this would start more than one Timer...or at least that's what happens to me for some reason so I get Tick invoked twice each time (two CPU cores).

I also noticed some weird behaviour. When trying to restart service with systemctl restart swoole it hangs for almost 2min and then it shouts that:

systemd[1]: swoole.service: State 'stop-sigterm' timed out. Killing.
systemd[1]: swoole.service: Killing process 707339 (php) with signal SIGKILL.
systemd[1]: swoole.service: Killing process 707342 (n/a) with signal SIGKILL.
systemd[1]: swoole.service: Killing process 707343 (n/a) with signal SIGKILL.
systemd[1]: swoole.service: Main process exited, code=killed, status=9/KILL

That happens only when all the stuff is in on('Start')...I tried setting it up in a different way but I couldn't get it properly because I'm getting "loop started" errors or the above.

Anyway, could you reopen this issue please? I'm really struggling here. :( I know I can simply start a new service, from a new file where the only thing would be Timer itself so there won't be any issues but I wanted to try and run that in the same place as API.

Edit: Maybe the question here should be: How to prevent on('WorkerStart') to execute more timers when I need only one but I do have more workers? Is it possible through some Swoole Magic? ;)

[deminy]

1. the onStart and onWorkerStart events run in different processes. onStart runs in the master process, and onWorkerStart runs in each every worker processes. So onStart happens once only when the server starts, while there could be many onWorkerStart events triggered, one for each worker process.
2. A better approach is to start the Timer in one of the worker processes (usually the first one), as suggested in this example.
3. I had a talk about how to "Build an All-In-One Application Server Using Swoole", which could help answering some of your questions. I'd suggest to take a look if you haven't yet. The talk starts at 20'55'', and you can find the slides here.
4. You can find more learning resources about Swoole from Awesome Swoole.

[Draghmar]

Thanks for the tips. I didn't know I could use worker ID this way. There so much Swoole's knowledge hidden...Those examples you've made should be easily accessible at least from Swoole's github main page. I was looking for some examples to the point I even was using translator on Chinese version. ;) Once again: big thanks for all the info! I will really put those into practice! :)