Huge VIP list results in session to not capture anything

[FlorianSW]

This might not be the case for most event servers, however, when looking at the general use cases this tool might to offer, I still want to raise the issue. Our server benefits from a VIP share program of big german clans, hence our VIP list for the server is massive (over 800 entries). This seems to make the log utilities to raise an error when trying to read the response:

hlllogutilities_1  | [2022-11-22 18:55:10,209][INFO][rcon.wrapper:39] Started RCON!
hlllogutilities_1  | [2022-11-22 18:55:10,259][ERROR][rcon.wrapper:81] Missed 1 consecutive updates for <lib.rcon.HLLRcon object at 0x7f9813a1f9a0>, 19 attempts left
hlllogutilities_1  | Traceback (most recent call last):
hlllogutilities_1  |   File "/code/lib/rcon.py", line 78, in wrapper
hlllogutilities_1  |     res = await asyncio.wait_for(func(self, *args, **kwargs), timeout=10)
hlllogutilities_1  |   File "/usr/local/lib/python3.8/asyncio/tasks.py", line 494, in wait_for
hlllogutilities_1  |     return fut.result()
hlllogutilities_1  |   File "/code/lib/rcon.py", line 155, in update
hlllogutilities_1  |     await self._fetch_server_info()
hlllogutilities_1  |   File "/code/lib/rcon.py", line 161, in _fetch_server_info
hlllogutilities_1  |     res = await asyncio.gather(
hlllogutilities_1  |   File "/code/utils.py", line 58, in wrapper
hlllogutilities_1  |     v = await func(*args, **kwargs)
hlllogutilities_1  |   File "/code/lib/rcon.py", line 389, in __fetch_player_roles
hlllogutilities_1  |     data = await self.exec_command('get vipids', unpack_array=True)
hlllogutilities_1  |   File "/code/lib/rcon.py", line 254, in exec_command
hlllogutilities_1  |     res = await fut
hlllogutilities_1  |   File "/code/lib/rcon.py", line 708, in _worker
hlllogutilities_1  |     res = await self.protocol.execute(cmd, **kwargs)
hlllogutilities_1  |   File "/code/lib/protocol.py", line 123, in execute
hlllogutilities_1  |     raise HLLUnpackError("Expected array size %s but got %s", arr_size, len(res))
hlllogutilities_1  | lib.exceptions.HLLUnpackError: ('Expected array size %s but got %s', 876, 41)
hlllogutilities_1  | [2022-11-22 18:55:15,259][ERROR][rcon.wrapper:81] Missed 2 consecutive updates for <lib.rcon.HLLRcon object at 0x7f9813a1f9a0>, 18 attempts left
hlllogutilities_1  | Traceback (most recent call last):
hlllogutilities_1  |   File "/code/lib/rcon.py", line 78, in wrapper
hlllogutilities_1  |     res = await asyncio.wait_for(func(self, *args, **kwargs), timeout=10)
hlllogutilities_1  |   File "/usr/local/lib/python3.8/asyncio/tasks.py", line 494, in wait_for
hlllogutilities_1  |     return fut.result()
hlllogutilities_1  |   File "/code/lib/rcon.py", line 155, in update
hlllogutilities_1  |     await self._fetch_server_info()
hlllogutilities_1  |   File "/code/lib/rcon.py", line 161, in _fetch_server_info
hlllogutilities_1  |     res = await asyncio.gather(
hlllogutilities_1  |   File "/code/utils.py", line 58, in wrapper
hlllogutilities_1  |     v = await func(*args, **kwargs)
hlllogutilities_1  |   File "/code/lib/rcon.py", line 389, in __fetch_player_roles
hlllogutilities_1  |     data = await self.exec_command('get vipids', unpack_array=True)
hlllogutilities_1  |   File "/code/lib/rcon.py", line 254, in exec_command
hlllogutilities_1  |     res = await fut
hlllogutilities_1  |   File "/code/lib/rcon.py", line 708, in _worker
hlllogutilities_1  |     res = await self.protocol.execute(cmd, **kwargs)
hlllogutilities_1  |   File "/code/lib/protocol.py", line 123, in execute
hlllogutilities_1  |     raise HLLUnpackError("Expected array size %s but got %s", arr_size, len(res))
hlllogutilities_1  | lib.exceptions.HLLUnpackError: ('Expected array size %s but got %s', 876, 41)

I think this is very much the same as #1, so not really much we can do about this issue (please correct me, if my analysis is wrong). However, what I wonder in the first place is: Why do we need to fetch the VIP status in the first place? The field (is_vip) is not really used anywhere, not even in the output logs (as far as I can see).

For the sake of simplicity and to make this working with our first server as well, I would simply remove this call to get the full vip list? Opinions?

[timraay]

Ah yeah, that is something I did not account for indeed. The real issue is that the game server does not tell us beforehand the amount of bytes the response will be in size. I suppose in this case we could tell whether the response is complete or not since we know the size of the array, but that's not something I ever really thought I needed in the first place.

To resolve the same issue with the showlog command, I simply wait a second for more packets to accumulate. The same could in theory be done here, by passing multipart=True to HLLRconProtocol.execute. But since we don't use the VIP information it might as well be skipped entirely.

[FlorianSW]

If we later need this information for something, it would be easy to re-add it, too 👍