UDP Sockets vs Stream Sockets?

[Ruok2bu]

Why is your method of native UDP sockets better than Austin's method stream sockets ?

[KostyaEsmukov]

Streams are heavy wrappers for sockets which provide the same usage for all of the protocols. However connectionless UDP and connection-oriented TCP protocols are very different. Native sockets are simple wrappers around BSD sockets, they are lower level. So they are (a little bit) faster.

UDP is connectionless protocol. It means that we can use single socket to speak with many servers at once. HLSW works like that. But HLSW uses only one UDP socket. GameQ3, due to it's queue style of packets (versus linear in HLSW) can send multiple packets to the same server and port at once. In this case we have to distinguish packets, so we create more sockets. So, whenever you have to request 10 or 5000 of servers, there will be created just about 3 sockets (an average count of packets in single queue) instead of 3_10 or 3_5000.

Due to lack of socket functions in streams, we can't just create a socket (we can do it using native sockets). To create a socket we have to call stream_socket_client() (which simulates connection to host) or stream_socket_server() (which simulates server and binds port). If we want to use method described above, the only good way to do it is to create stream_socket_server() and use stream_socket_recvfrom() with stream_socket_sendto().

So, it is possible to do the same using streams, but it will be slower. Also, look at the https://github.com/kostya0shift/GameQ3/blob/master/gameq3/sockets.php file, you'll find a note about stream_select bug in PHP 5.3 which slows processing.

As stated in README, I want to make GameQ3 as fast as it is possible, so I decided to use native sockets for UDP as for most frequent protocol in game servers querying. All other protocols (except HTTP(S)) are processed by streams. For example, 670 source servers (ping from 20 to 300 ms) are being requested in around 4.7 seconds. I'll commit longlist example soon, I'm very busy for now.

I hope that I had answered your question.

[KostyaEsmukov]

If there will be an interest to this library, I'll add some documentation about how does it works and about advanced usage.

[Ruok2bu]

Did the definitions for querying servers change with your fork or is it just the underlying code (eg. i can just install the files and my existing query codes will keep on working). Also is it just the core files that are changed or are the protocols changed too? (can i use protocols from v2?)

[KostyaEsmukov]

I'm sorry for replying late because I was really busy last week. Of course libraries are very different. If you want to move from v2 to GameQ3, then you should notice that public methods are a little bit different, options are totally different, response is also different. If you use calls like $gq->func1()->func2(); , then you should change them to $gq->func1(); $gq->func2(); , because I don't see any practical usage for chains in case of GameQ. Protocols has changed very much (due to usage of queue style of packets) so they are not compatible. Simple example: minecraft; complicated examples: source, gamespy, lfs.

Here is a short guide to move your query software from V2 to GameQ3:

Public methods

Functions for setting up options are the same (magics added in last commit), but options are different (see below).

• get -> get
• set -> set
• setOption -> setOption
• - -> setLogLevel($error, $warning = true, $debug = false, $trace = false)
• - -> setLogger($callback)

Filter functions names are a little bit changed. Filters are also different, see below.

• setFilter -> setFilter
• removeFilter -> unsetFilter

Working with servers list is totally different. There is no addServers method in GameQ3, because you have to decide what to do when exception occurs. Please refer to examples to see how it works. Server info array is also changed, see below.

• addServer -> addServer
• addServers -> -
• clearServers -> -
• - -> unsetServer

Request functions (Basically you should use only requestAllData. You can use requestPartData to save memory if you have to query very large amount of servers):

• requestData -> requestAllData
• - -> requestPartData

Options:

V2:

• debug // replace it for GameQ3 to setLogLevel method
• timeout _// replace it with readtimeout and change seconds to miliseconds
• filters // replace it with filters functions
• streamtimeout // replace it with connecttimeout and change microseconds to miliseconds

GameQ3 : Timeout options (all must be integers, in miliseconds):

• connect_timeout
• read_timeout
• read_gottimeout - how much time to wait for the next packet_
• read_retrytimeout - timeout for retries. It is separate, because the server you try to query is probably offline_
• looptimeout - pause between bunches of packets send_
• curl_selecttimeout - timeout for curl_multiselect function
• curl_connect_timeout
• curl_total_timeout

Quantities:

• serverscount - number of servers to query at once. Related to memory usage, request time. requestPartData will return not more than this number of servers_
• send_onceudp - how many packets to send in single bunch. We need this to avoid flooding servers we request_
• send_oncestream - the same for streams_
• sendretry - quantity of retry attempts_

Pauses (nanoseconds):

• usleepudp - how many time to sleep after udp bunch of sends_
• usleepstream - the same for streams_

Other:

• socketbuffer - bytes_
• curloptions - array of curl_setopt settings (see php.net curl_setopt)_

Filters:

• // Normalization is built-in, so:
• normalise -> -
• stripcolor -> colorize with argument 'strip' (see /examples/index.php)
• - -> sortplayers (see /examples/index.php)

Server info array (an argument for addServer method):

V2 to GameQ3:

• id -> id (required!!!)
• type -> type (some protocol names are changed, list of protocols is here: /gameq3/protocols)
• host -> query_host
• options -> - (options for protocols are set directly in server info array)

New GameQ3 keys:

• filters _- array of filters for that server in format filter_name => filter_arguments. filterarguments must be an array or boolean false
• queryhost - examples: '127.0.0.1:27015', '[::1]:27015', 'localhost:27015'_
• queryaddr - examples: '127.0.0.1', '[::1]', 'localhost'_
• queryport - integer port. example: 27015_
• connect_host, connect_addr, connect_port are set in the same manner. The difference between them is pretty simple: some servers have query port different from connect port. GameQ3 tries (when query port is not set) to guess query port by connect port when it is possible
• debug - raise log priority from debug to warning for this server. (see /gameq3/log.php)
• unset - array of response keys to be excluded. Very handy if you need to request large amount of servers just for general data. It will save your memory and additional requests when possible
• ... - any other key will be passed to protocols, so they can use them. Example protocols using custom options: lfs, teamspeak3

Please launch /examples/index.php to see how response looks. Feel free to ask more questions.

[Ruok2bu]

How much of a speed boost (percentage wise approximation) does your method of UDP Sockets give versus austin's stream sockets?

[KostyaEsmukov]

Just try by yourself and decide what you like more. Sockets implementation is just one of the differences.

I took 500 l4d2 servers from gametracker top (nearest to me). GameQ v2 by default has timeout 200ms, GameQ3 - 600 ms. So to make tests clean, I changed timeout of GameQ3 to 200 and performed 2 runs: with send_retry=1 (default) and without retries. Online servers count is variable, because when server changes the map, it goes offline for some time. I don't know why, but in GameQ v2 last 100-200 servers are always offline. Maybe it is a bug, I have no time to test is for now. Here are the results of my tests (first - GameQ3, second - GameQ v2:

---

_readtimeout=200 Online: 185. Offline: 15. Retrys: 3. Total: 200. Ping avg: 74.85ms. Time elapsed: 1.5596840381622s. Memory result/unset/max (kb) 18216/1335/26046

Online: 185. Offline: 15. Total: 200. Time elapsed: 1.6103069782257s. Memory result/unset/max (kb) 17369/10836/17608

---

_read_timeout=200 sendretry=0 Online: 181. Offline: 19. Retrys: 0. Total: 200. Ping avg: 75.82ms. Time elapsed: 1.1664600372314s. Memory result/unset/max (kb) 17739/1336/25398

Online: 182. Offline: 18. Total: 200. Time elapsed: 1.6066811084747s. Memory result/unset/max (kb) 16919/10556/17159

---

_readtimeout=200 Online: 469. Offline: 31. Retrys: 56. Total: 500. Ping avg: 84.89ms. Time elapsed: 2.6109659671783s. Memory result/unset/max (kb) 39763/1974/57899

Online: 329. Offline: 171. Total: 500. Time elapsed: 2.1343359947205s. Memory result/unset/max (kb) 30575/19622/30862

---

_read_timeout=200 sendretry=0 Online: 462. Offline: 38. Retrys: 0. Total: 500. Ping avg: 85.60ms. Time elapsed: 2.28005194664s. Memory result/unset/max (kb) 39403/1977/57380

Online: 299. Offline: 201. Total: 500. Time elapsed: 2.1179320812225s. Memory result/unset/max (kb) 29539/18966/29839

[Ruok2bu]

Hmm, according to those results there isin't a significant performance boost.

[KostyaEsmukov]

You won't notice significant speed boost on small amounts of servers due to heavy wrappers around sockets. But they are much customisable, letting you use optimal settings for your network (whenever it is fast and stable or not) and servers count. All these playings with settings are for stability. Default GameQ3 settings are the average between speed and stability. If all you care about is speed, then you can set hard limits (compare with the last test from my previous comment):

_read_timeout=200 read_got_timeout=5 usleep_udp=80 loop_timeout=1 sendretry=0 Online: 309. Offline: 191. Retrys: 0. Total: 500. Ping avg: 82.42ms. Time elapsed: 1.6654489040375s. Memory result/unset/max (kb) 28589/2011/42479

But does really speed matters more than stability for you? As for me - stability is the first. That's the reason for having so much settings which adapt GameQ3 for different networks and game servers (some of them don't like too fast packets, source is one of them). The killer feature is send_retry setting. It lets GameQ3 to re-request offline servers to make sure they are offline. Very useful.