search

Yepoleb / python-a2s

A2S Query API for Python
MIT License
154 stars 17 forks source link

Timeout even after receiving some data #16

Closed Nereg closed 3 years ago

Nereg commented 4 years ago

Hello ! I spotted very strange things going on. So I tested server with IP 95.156.213.151:27017 (official ARK:SE server) and got such results:

$ py test.py
Received single packet: b'I\x11NA-PVP-Official-CrystalIsles808 - (v313.33)\x00CrystalIsles\x00ark_survival_evolved\x00ARK: Survival Evolved\x00\x00\x00\x13F\x00dw\x00\x011.0.0.0\x00\xb1c\x1e\x03<\x17\xdb\xd7<@\x01,OWNINGID:90138890338778115,OWNINGNAME:90138890338778115,NUMOPENPUBCONN:61,P2PADDR:90138890338778115,P2PPORT:7779,LEGACY_i:0\x00\xfeG\x05\x00\x00\x00\x00\x00'
Received single packet: b'A\xd7lT\t'
Received single packet: b'D\x13\x00\x00\x00\x00\x00\x00\xe3QLH\x00\x00\x00\x00\x00\x00\x12\x8eGH\x00\x00\x00\x00\x00\x00\x95\x8bEH\x00\x00\x00\x00\x00\x008\xf6<H\x00\x00\x00\x00\x00\x00\xb494H\x00\x00\x00\x00\x00\x00\x90\xe7,H\x00\x00\x00\x00\x00\x00\xce(\x90G\x00\x00\x00\x00\x00\x00\xde\xf9\x89G\x00\x00\x00\x00\x00\x00(\xfc\x83G\x00\x00\x00\x00\x00\x00\xbf\x9cqG\x00\x00\x00\x00\x00\x00\x7f\nKG\x00YA..\x00\x00\x00\x00\x00\xc6\xaf,F\x00123\x00\x00\x00\x00\x00\x08\x91\x12F\x00BlueEternal\x00\x00\x00\x00\x00uI\xa8E\x00123\x00\x00\x00\x00\x00\xb2\xeflE\x00awesome neon\x00\x00\x00\x00\x00id^E\x00kai\x00\x00\x00\x00\x00\xf3\x1c5E\x00123\x00\x00\x00\x00\x00\xf6\x1a,E\x00Davos[373]\x00\x00\x00\x00\x00\xd6\x03\x87D'
Traceback (most recent call last):
  File "test.py", line 19, in <module>
    data = asyncio.get_event_loop().run_until_complete(a2s.arules(address))
  File "c:\users\oleg\appdata\local\programs\python\python37-32\lib\asyncio\base_events.py", line 579, in run_until_complete
    return future.result()
  File "D:\Projects\Current\ARK Server monitor\lib\site-packages\a2s\rules.py", line 63, in arules
    reader = await rules_request_async(conn, encoding)
  File "D:\Projects\Current\ARK Server monitor\lib\site-packages\a2s\rules.py", line 68, in rules_request_async
    resp_data = await conn.request(b"\x56" + challenge.to_bytes(4, "little"))
  File "D:\Projects\Current\ARK Server monitor\lib\site-packages\a2s\a2sasync.py", line 92, in request
    return await self.recv()
  File "D:\Projects\Current\ARK Server monitor\lib\site-packages\a2s\a2sasync.py", line 86, in recv
    raise asyncio.TimeoutError()
concurrent.futures._base.TimeoutError

How it could be posible ? And by the way it only happens with async implementation. With sync one script finishes just fine. And you can find my script here Also here is output of the same script with sync calls:

$ py test.py
Received single packet: b'I\x11NA-PVP-Official-CrystalIsles808 - (v313.33)\x00CrystalIsles\x00ark_survival_evolved\x00ARK: Survival Evolved\x00\x00\x00\x11F\x00dw\x00\x011.0.0.0\x00\xb1c\x1e\x03<\x17\xdb\xd7<@\x01,OWNINGID:90138890338778115,OWNINGNAME:90138890338778115,NUMOPENPUBCONN:62,P2PADDR:90138890338778115,P2PPORT:7779,LEGACY_i:0\x00\xfeG\x05\x00\x00\x00\x00\x00'
Received single packet: b'A\xd7lT\t'
Received single packet: b'D\x11\x00\x00\x00\x00\x00\x00H\x9fLH\x00\x00\x00\x00\x00\x00w\xdbGH\x00\x00\x00\x00\x00\x00\xf9\xd8EH\x00\x00\x00\x00\x00\x00\x9dC=H\x00\x00\x00\x00\x00\x00\x19\x874H\x00\x00\x00\x00\x00\x00\xf44-H\x00\x00\x00\x00\x00\x00\x97\xc3\x90G\x00\x00\x00\x00\x00\x00\xa8\x94\x8aG\x00\x00\x00\x00\x00\x00Q\xd2rG\x00\x00\x00\x00\x00\x00\x11@LG\x00YA..\x00\x00\x00\x00\x00\x11\x861F\x00123\x00\x00\x00\x00\x00Sg\x17F\x00BlueEternal\x00\x00\x00\x00\x00\x0b\xf6\xb1E\x00awesome neon\x00\x00\x00\x00\x00\x94\xbdqE\x00123\x00\x00\x00\x00\x00"t?E\x00Davos[373]\x00\x00\x00\x00\x00,\xb6\xadD\x00\x00\x00\x00\x00\x00\xd0\x86KB'
Received single packet: b'A\xd7lT\t'
Received single packet: b'E\x16\x00ALLOWDOWNLOADCHARS_i\x001\x00ALLOWDOWNLOADITEMS_i\x001\x00ClusterId_s\x00NewPCPVP\x00CUSTOMSERVERNAME_s\x00na-pvp-official-crystalisles808\x00DayTime_s\x002394\x00GameMode_s\x00TestGameMode_C\x00HASACTIVEMODS_i\x000\x00LEGACY_i\x000\x00MATCHTIMEOUT_f\x00120.000000\x00ModId_l\x000\x00Networking_i\x000\x00NUMOPENPUBCONN\x0062\x00OFFICIALSERVER_i\x001\x00OWNINGID\x0090138890338778115\x00OWNINGNAME\x0090138890338778115\x00P2PADDR\x0090138890338778115\x00P2PPORT\x007779\x00SEARCHKEYWORDS_s\x00Custom\x00ServerPassword_b\x00false\x00SERVERUSESBATTLEYE_b\x00true\x00SESSIONFLAGS\x001707\x00SESSIONISPVE_i\x000\x00'
{'ALLOWDOWNLOADCHARS_i': '1', 'ALLOWDOWNLOADITEMS_i': '1', 'ClusterId_s': 'NewPCPVP', 'CUSTOMSERVERNAME_s': 'na-pvp-official-crystalisles808', 'DayTime_s': '2394', 'GameMode_s': 'TestGameMode_C', 'HASACTIVEMODS_i': '0', 'LEGACY_i': '0', 'MATCHTIMEOUT_f': '120.000000', 'ModId_l': '0', 'Networking_i': '0', 'NUMOPENPUBCONN': '62', 'OFFICIALSERVER_i': '1', 'OWNINGID': '90138890338778115', 'OWNINGNAME': '90138890338778115', 'P2PADDR': '90138890338778115', 'P2PPORT': '7779', 'SEARCHKEYWORDS_s': 'Custom', 'ServerPassword_b': 'false', 'SERVERUSESBATTLEYE_b': 'true', 'SESSIONFLAGS': '1707', 'SESSIONISPVE_i': '0'}
SourceInfo(protocol=17, server_name='NA-PVP-Official-CrystalIsles808 - (v313.33)', map_name='CrystalIsles', folder='ark_survival_evolved', game='ARK: Survival Evolved', app_id=0, player_count=17, max_players=70, bot_count=0, server_type='d', platform='w', password_protected=False, vac_enabled=True, version='1.0.0.0', edf=177, port=7779, steam_id=90138890338778115, stv_port=None, stv_name=None, keywords=',OWNINGID:90138890338778115,OWNINGNAME:90138890338778115,NUMOPENPUBCONN:62,P2PADDR:90138890338778115,P2PPORT:7779,LEGACY_i:0', game_id=346110, ping=0.1720000000204891)
.... here goes my own output...

So server's response for player request is different: Async version: b'D\x13\x00\x00\x00\x00\x00\x00\xe3QLH\x00\x00\x00\x00\x00\x00\x12\x8eGH\x00\x00\x00\x00\x00\x00\x95\x8bEH\x00\x00\x00\x00\x00\x008\xf6<H\x00\x00\x00\x00\x00\x00\xb494H\x00\x00\x00\x00\x00\x00\x90\xe7,H\x00\x00\x00\x00\x00\x00\xce(\x90G\x00\x00\x00\x00\x00\x00\xde\xf9\x89G\x00\x00\x00\x00\x00\x00(\xfc\x83G\x00\x00\x00\x00\x00\x00\xbf\x9cqG\x00\x00\x00\x00\x00\x00\x7f\nKG\x00YA..\x00\x00\x00\x00\x00\xc6\xaf,F\x00123\x00\x00\x00\x00\x00\x08\x91\x12F\x00BlueEternal\x00\x00\x00\x00\x00uI\xa8E\x00123\x00\x00\x00\x00\x00\xb2\xeflE\x00awesome neon\x00\x00\x00\x00\x00id^E\x00kai\x00\x00\x00\x00\x00\xf3\x1c5E\x00123\x00\x00\x00\x00\x00\xf6\x1a,E\x00Davos[373]\x00\x00\x00\x00\x00\xd6\x03\x87D' Sync version: b'D\x11\x00\x00\x00\x00\x00\x00H\x9fLH\x00\x00\x00\x00\x00\x00w\xdbGH\x00\x00\x00\x00\x00\x00\xf9\xd8EH\x00\x00\x00\x00\x00\x00\x9dC=H\x00\x00\x00\x00\x00\x00\x19\x874H\x00\x00\x00\x00\x00\x00\xf44-H\x00\x00\x00\x00\x00\x00\x97\xc3\x90G\x00\x00\x00\x00\x00\x00\xa8\x94\x8aG\x00\x00\x00\x00\x00\x00Q\xd2rG\x00\x00\x00\x00\x00\x00\x11@LG\x00YA..\x00\x00\x00\x00\x00\x11\x861F\x00123\x00\x00\x00\x00\x00Sg\x17F\x00BlueEternal\x00\x00\x00\x00\x00\x0b\xf6\xb1E\x00awesome neon\x00\x00\x00\x00\x00\x94\xbdqE\x00123\x00\x00\x00\x00\x00"t?E\x00Davos[373]\x00\x00\x00\x00\x00,\xb6\xadD\x00\x00\x00\x00\x00\x00\xd0\x86KB' I would dive deeper and view what your library is sending in those two cases. Also I would analyse responses to find if it can be parsed anyway.

Nereg commented 4 years ago

Tested one more time with async and even shorter output:

$ py test.py
Received single packet: b'I\x11NA-PVP-Official-CrystalIsles808 - (v313.33)\x00CrystalIsles\x00ark_survival_evolved\x00ARK: Survival Evolved\x00\x00\x00\x15F\x00dw\x00\x011.0.0.0\x00\xb1c\x1e\x03<\x17\xdb\xd7<@\x01,OWNINGID:90138890338778115,OWNINGNAME:90138890338778115,NUMOPENPUBCONN:54,P2PADDR:90138890338778115,P2PPORT:7779,LEGACY_i:0\x00\xfeG\x05\x00\x00\x00\x00\x00'
Received single packet: b'A\xf5 \x0c\x05'
Traceback (most recent call last):
  File "test.py", line 21, in <module>
    players = asyncio.get_event_loop().run_until_complete(a2s.aplayers(address))
  File "c:\users\oleg\appdata\local\programs\python\python37-32\lib\asyncio\base_events.py", line 579, in run_until_complete
    return future.result()
  File "D:\Projects\Current\ARK Server monitor\lib\site-packages\a2s\players.py", line 72, in aplayers
    reader = await players_request_async(conn, encoding)
  File "D:\Projects\Current\ARK Server monitor\lib\site-packages\a2s\players.py", line 88, in players_request_async
    conn, encoding, challenge, retries + 1)
  File "D:\Projects\Current\ARK Server monitor\lib\site-packages\a2s\players.py", line 77, in players_request_async
    resp_data = await conn.request(b"\x55" + challenge.to_bytes(4, "little"))
  File "D:\Projects\Current\ARK Server monitor\lib\site-packages\a2s\a2sasync.py", line 92, in request
    return await self.recv()
  File "D:\Projects\Current\ARK Server monitor\lib\site-packages\a2s\a2sasync.py", line 86, in recv
    raise asyncio.TimeoutError()
concurrent.futures._base.TimeoutError

P.S. Runed one more time and it finished like normal ... what is going on ?! P.P.S Got exception even without any output

Nereg commented 4 years ago

Modified library to print all packets sent. It ran servial times successfully but then that happend :

$ py test.py
Made connection!
Sent packet:b'\xff\xff\xff\xffTSource Engine Query\x00'
Received single packet: b'I\x11NA-PVP-Official-CrystalIsles808 - (v313.33)\x00CrystalIsles\x00ark_survival_evolved\x00ARK: Survival Evolved\x00\x00\x00\x14F\x00dw\x00\x011.0.0.0\x00\xb1c\x1e\x03<\x17\xdb\xd7<@\x01,OWNINGID:90138890338778115,OWNINGNAME:90138890338778115,NUMOPENPUBCONN:55,P2PADDR:90138890338778115,P2PPORT:7779,LEGACY_i:0\x00\xfeG\x05\x00\x00\x00\x00\x00'
Made connection!
Sent packet:b'\xff\xff\xff\xffU\x00\x00\x00\x00'
Received single packet: b'A\xf5 \x0c\x05'
Sent packet:b'\xff\xff\xff\xffU\xf5 \x0c\x05'
Traceback (most recent call last):
  File "test.py", line 21, in <module>
    players = asyncio.get_event_loop().run_until_complete(a2s.aplayers(address))
  File "c:\users\oleg\appdata\local\programs\python\python37-32\lib\asyncio\base_events.py", line 579, in run_until_complete
    return future.result()
  File "D:\Projects\Current\ARK Server monitor\lib\site-packages\a2s\players.py", line 72, in aplayers
    reader = await players_request_async(conn, encoding)
  File "D:\Projects\Current\ARK Server monitor\lib\site-packages\a2s\players.py", line 88, in players_request_async
    conn, encoding, challenge, retries + 1)
  File "D:\Projects\Current\ARK Server monitor\lib\site-packages\a2s\players.py", line 77, in players_request_async
    resp_data = await conn.request(b"\x55" + challenge.to_bytes(4, "little"))
  File "D:\Projects\Current\ARK Server monitor\lib\site-packages\a2s\a2sasync.py", line 94, in request
    return await self.recv()
  File "D:\Projects\Current\ARK Server monitor\lib\site-packages\a2s\a2sasync.py", line 88, in recv
    raise asyncio.TimeoutError()
concurrent.futures._base.TimeoutError

Maybe it could be fixed by raising timeout? P.S. No. Even 10 seconds isn't enough.

Nereg commented 4 years ago

I think it is one more bug/lag in ARK:SE. I just hate it and it's developers

Nereg commented 4 years ago

But maybe I can make mod to raise different exeption to just don't mark this server as offline and try again later. I'll pull it here

Nereg commented 4 years ago

Also it maybe some anti-DDOS cap. I will count how much packets I must send before getting banned.

Nereg commented 4 years ago

Also same happens for sync version:

Traceback (most recent call last):
  File "test.py", line 22, in <module>
    server = a2s.info(address)
  File "D:\Projects\Current\ARK Server monitor\lib\site-packages\a2s\info.py", line 267, in info
    resp_data = conn.request(b"\x54Source Engine Query\0")
  File "D:\Projects\Current\ARK Server monitor\lib\site-packages\a2s\a2sstream.py", line 50, in request
    return self.recv()
  File "D:\Projects\Current\ARK Server monitor\lib\site-packages\a2s\a2sstream.py", line 28, in recv
    packet = self._socket.recv(65535)
socket.timeout: timed out

Packets sent:15
Nereg commented 4 years ago

Yeah it seems like DDOS-protection.I sent 15 packets and got baned then waited and sent smt like 50 and no problem. It also will explain why I got strange errors that I couldn't remake on my own machine.

Yepoleb commented 4 years ago

You can parse the players response manually like this:

import io
from a2s.players import players_response
from a2s.byteio import ByteReader
packet_data = b'D\x13\x00\x00\x00\x00\x00\x00\xe3QLH\x00\x00\x00\x00\x00\x00\x12\x8eGH\x00\x00\x00\x00\x00\x00\x95\x8bEH\x00\x00\x00\x00\x00\x008\xf6<H\x00\x00\x00\x00\x00\x00\xb494H\x00\x00\x00\x00\x00\x00\x90\xe7,H\x00\x00\x00\x00\x00\x00\xce(\x90G\x00\x00\x00\x00\x00\x00\xde\xf9\x89G\x00\x00\x00\x00\x00\x00(\xfc\x83G\x00\x00\x00\x00\x00\x00\xbf\x9cqG\x00\x00\x00\x00\x00\x00\x7f\nKG\x00YA..\x00\x00\x00\x00\x00\xc6\xaf,F\x00123\x00\x00\x00\x00\x00\x08\x91\x12F\x00BlueEternal\x00\x00\x00\x00\x00uI\xa8E\x00123\x00\x00\x00\x00\x00\xb2\xeflE\x00awesome neon\x00\x00\x00\x00\x00id^E\x00kai\x00\x00\x00\x00\x00\xf3\x1c5E\x00123\x00\x00\x00\x00\x00\xf6\x1a,E\x00Davos[373]\x00\x00\x00\x00\x00\xd6\x03\x87D'
packet_stream = io.BytesIO(packet_data)
packet_reader = ByteReader(packet_stream)
packet_reader.read_uint8() # skip response type
print(players_response(packet_reader))

Adding more debugging output is a good idea, I will do that (see #17). For the timeouts I don't think this is a bug in the library, the application is responsible of keeping enough time between requests, so I'm not sure what to do here.

Nereg commented 4 years ago

Yeah I modified my app to keep time from request to request. But it should be noted in readme I think. And I would test how much I can send before I get banned.