Need for more than one server connection

[botorabi]

Background I am using this awesome lib for a bot server framework running with php cli, i.e. not in context of a web server such as apache, for more details see https://github.com/botorabi/TeamSpeakPHPBots

So far I could see, the lib is usually used for establishing one single connection to ts3 server query. However, sometimes there is an urgent need for multiple connections.

Current Situation The library provides means for creating a new connection to a ts3 server query and starting a rather event driven communication. In simplest form this looks like following.

1) establish connection in non-blocking mode 2) begin main loop consisting of 2.1) wait for an event by calling TeamSpeak3_Adapter_ServerQuery's wait() method 2.2) optionally one can perform useful things in the context of "timeout" events.

What is missing The wait() method mentioned above waits 'infinite' until an event arrives or the connection is interrupted, so there is no chance for polling multiple connections. What is missing is the possibility that the wait method returns after a defined timeout, instead of waiting for an event arrival. Thus it would be possible to poll multiple connections in the main application loop and do also app specific periodic tasks.

Conclusion Maybe I have missed some features allowing me exactly what I need (managing multiple connections in the same php application and poll them in app's main loop), but I fear the current design is not meant to be used in such a way. Any hints on this topic are highly appreciated. If this feature seems useful also to you, then I could adapt the library and create a pull request.

[botorabi]

I have done some minimal extensions to the library and voila, it works now fine with multiple connections. Just apply the attached patch to version 1.1.24.

What does the patch do? The patch brings a timeout parameter to server query's wait method: '$waitForEver'. In addition, a new querytext parameter is supported: 'readtimeout'. The existing 'timeout' parameter is further used for establishing the connection and reading as usual. But if you call wait with $waitForEver set to false then the 'readtimeout' is used which is given in milliseconds.

What does the patch not do? It does not break the existing ABI. Its functionality is fully optional.

Example Here is an example for the new functionality the provided patch allows.

// your main application loop // establish several connections to ts3 server with 'readtimeout' set to say 500 while(not teminate the app) { $server1->getAdapter()->wait(false); $server2->getAdapter()->wait(false); $server3->getAdapter()->wait(false); ... do other fancy stuff in your app }

Now you can iterate over your connections and poll them, given every poll a maximal timeout (here 500 milliseconds).

TeamSpeak3-v1.1.24-patch_multiple_connections.patch.zip

[svenpaulsen]

This is indeed a major design flaw and I'm addressing it in version 2 of the framework. I'll have a look at your patch and merge it into the master branch if everything works. Thanks for the report!

[botorabi]

It sounds great, Sven!

[ronindesign]

This is an issue of asynchronicity and should be handled on a socket management level where scanning can occur without setting an arbitrary timeout length.

A loop manager would handle checking for new activity. I've implemented a few different ways of doing this and will post a couple examples. It would be great to get some of this functionality added to the framework so it's available natively.

Some pseudo code such as:

$loop = \EventLoop\Factory::create();

$server1 = new TeamSpeak3::getReactiveInstance($loop);
$server2 = new TeamSpeak3::getReactiveInstance($loop);

$server1->on(
    'clientmoved',
    function($event) {
        sprintf( date('Y-m-d H:i:s) . " Client (%s) moved", $event->clid );
    }
);

$server2->on(
    'cliententerview',
    function($event) {
        sprintf( date('Y-m-d H:i:s) . " Client (%s) conected", $event->clid );
    }
);

$loop->run;

This way the scanning is all managed in a separate, decoupled layer that can be scaled, workers or tasks forked to child processes, etc.

Examples of existing realworld async I/O implementations are reactphp, ratchet/websocks, php-zeromq, native socket_stream_select watchers.

[botorabi]

I have a slightly better patch which takes a different argument for the server query's wait method: a timeout value in milliseconds. This allows to have individual timeouts for your connections instead of one global timeout used for all connections. This turned out to be better in dealing with my bots' query server connections. Just apply it to version 1.1.14.

TeamSpeak3-v1.1.24-patch_multiple_connections.patch-II.zip

[mqxym]

I also faced this problem and "solved" it by doing multithreading in PHP using the pthreads extension. Each connection takes part in a different thread, therefore it is possible to create event-driven bots but also interval-driven bots (for example check every client every 5 seconds for idle time).

[botorabi]

Hi Wackelkontakt,

unfortunately, I had no big luck using the multi-threaded version of PHP in my project. Beside the additional overhead in initial setup and maintenance for having the multi-threaded version on our debian servers, it turned out that this PHP version introduced a couple of runtime problems such as issues with static variables in PHP classes -- I ran into problems using this framework in several threads as also this framework uses static variables. While I think the multi-threaded version of PHP is really something what I would like to use in order to implement usual CLI based services needing a degree of concurrent tasks, I fear the assumptions this version puts into implementation of your program is something not compatible with a lot of existing PHP libraries.

[JABirchall]

@ronindesign I personally don't like that recommendation as it removes the freedom of modularity that the current version allows. take for example my bot I wrote where its main design is modularity: https://github.com/JABirchall/NimdaTS3/blob/master/app/TeamSpeak3Bot.php

Instead I would suggest to keep the notifyEvent, notifyTextmessage etc etc since these events are documented and registered on the TeamSpeak server its self.

[ronindesign]

@JABirchall My example was for showing how the underlying socket layer scanning would occur.. A user of the framework would still interface the same way, using notify register events, etc.

The difference is being able to define multiple $ts3_HostServer = TeamSpeak3::factory(...) instances and have them registered to a single loop engine. All the actual usage of the TeamSpeak3 framework would be the same, it would just support multiple routes.

I guess a practical example in terms of events would be something like:

require_once 'TeamSpeak3.php';
$ts3WelcomeBot = TeamSpeak3::factory("serverquery://serveradmin:password@127.0.01:10011/?nickname=WelcomeBot&server_port=9987&blocking=0");
TeamSpeak3_Helper_Signal::getInstance()->subscribe("notifyCliententerview", "onCliententerview");
$ts3WelcomeBot->notifyRegister("server");
$ts3WelcomeBot->getAdapter()->wait();

$ts3AuditBot = TeamSpeak3::factory("serverquery://serveradmin:password@127.0.01:10011/?nickname=AuditBot&server_port=9987&blocking=0");
TeamSpeak3_Helper_Signal::getInstance()->subscribe("notifyServeredited", "onServeredited");
$ts3AuditBot->notifyRegister("server");
$ts3AuditBot->getAdapter()->wait();

$ts3ChatBot = TeamSpeak3::factory("serverquery://serveradmin:password@127.0.01:10011/?nickname=ChatBot&server_port=9987&blocking=0");
TeamSpeak3_Helper_Signal::getInstance()->subscribe("notifyTextmessage", "onTextmessage");
$ts3ChatBot->notifyRegister("server");

Now instead of doing the typical: $ts3ChatBot->getAdapter()->wait(); or while(1) { $ts3ChatBot->getAdapter()->wait(); // do stuff }

You would simply pass the loop handling off to a socket manager, something like:

$loop = \EventLoop\Factory::create();
$loop->manage($ts3WelcomeBot);
$loop->manage($ts3AuditBot);
$loop->manage($ts3ChatBot);
$loop->run();

So my point being, you don't actually have to do any different logic, per instance. You just pass reference off to a manager to handle socket scanning, which then simply returns focus back to whichever instance has active buffer, etc.

I reviewed your bot code, and internally it looks like it still blocks on the TeamSpeak3_Adapter_ServerQuery->wait() logic:

    /**
     * Wait for messages
     */
    protected function wait()
    {
        while ($this->online === true) {
            $this->node->getAdapter()->wait();
        }
    }

If instead you had an array of nodes or connections that you were scanning over in your TeamSpeak3Bot->wait() and checking for active (unread) buffer, then this would be more comparable to what we're talking about. I apologize if I missed some other logic in your code that implements this multiple connection handling!

[JABirchall]

I just stumbled upon this async network framework, Could be a good starting point: https://github.com/walkor/Workerman

[svenpaulsen]

I know that lib... The problem is, that it does not work on Windows based webservers. Also, PHP should not be the first choice for async programming.

[ronindesign]

It seems like the language choice for async implementation, at least in this case, would not be that large of a factor. Higher-level languages (python, php, etc) offload their heavy work to c/c++ compiled libraries anyways, even further they can be compiled down or derived (cython, phalcon), but again this minor optimization wouldn't seem to balance out the cost of less accessible code.

I feel like that is the main benefit of PHP (again, used in this case): to develop a decently performant model that meets required functionality, yet is still open and accessible enough to the general audience of developers (i.e. most likely web developers).

Further, if optimization is a priority, improving implementation and algorithms would far out way the difference in language (when choosing high-level language).

Would you recommend something statically typed such as java instead? Or a different dynamic such as javascript or python?

[svenpaulsen]

I'd use Python...

[ronindesign]

That would make sense, Python is pretty flexible... thanks!

[JABirchall]

Well you could also use pthreads it works on linux and windows. and you can check if the extension is loaded or not and throw an exception if someone tried to do multi-connections without pthreads loaded.

[JABirchall]

Also just thought about PCNTL you can use pcmtl_folk to create a child process and pcntl_wait to wait for input

[JABirchall]

In the art of providing information, I have just discovered package: https://github.com/bwoebi/php-uv

looks like EXACTLY what is needed for the framework to have concurrent tasks in PHP.

Edit: also found this package which utilises extensions like the following to achieve the task: https://github.com/amphp/amp/tree/v2.0.0/