991jo / srcds_exporter

an prometheus exporter for SRCDS based gameservers (like CSGO, TF2, L4D2, ...).
17 stars 5 forks source link

Error while running script #9

Closed larrryfisherthefisherman closed 5 years ago

larrryfisherthefisherman commented 5 years ago

While the script is running it will throw this out every so often. I am running the scraper every 5s like your example. I also have 14 servers set to scrape in this way in the prometheus.yml:

        - targets: ["172.217.165.14:27015:password"] 
        - targets: ["172.217.165.14:27020:password"] 
[14/04/2019 18:14] ERROR test handler 135:
Traceback (most recent call last):
  File "test.py", line 81, in handler
    status, stats = await rcon_query(ip, port, password)
  File "test.py", line 58, in rcon_query
    2)
  File "/usr/lib/python3.5/asyncio/tasks.py", line 404, in wait_for
    raise futures.TimeoutError()
concurrent.futures._base.TimeoutError

as well as:

 Traceback (most recent call last):
  File "main.py", line 82, in handler
    status, stats = await rcon_query(ip, port, password)
  File "main.py", line 59, in rcon_query
    2)
  File "/usr/lib/python3.5/asyncio/tasks.py", line 400, in wait_for
    return fut.result()
  File "/usr/lib/python3.5/asyncio/futures.py", line 293, in result
    raise self._exception
  File "/usr/lib/python3.5/asyncio/tasks.py", line 241, in _step
    result = coro.throw(exc)
  File "/home/test/.local/lib/python3.5/site-packages/aiorcon/rcon.py", line 32, in create
    await rcon._connect()
  File "/home/test/.local/lib/python3.5/site-packages/aiorcon/rcon.py", line 37, in _connect
    await protocol.authenticate()
  File "/home/test/.local/lib/python3.5/site-packages/aiorcon/protocol.py", line 40, in wrapper
    return await func(instance, *args, **kwargs)
  File "/home/test/.local/lib/python3.5/site-packages/aiorcon/protocol.py", line 78, in authenticate
    res = await self._receive()
  File "/home/test/.local/lib/python3.5/site-packages/aiorcon/protocol.py", line 99, in _receive
    await self._waiters[id_]
  File "/usr/lib/python3.5/asyncio/futures.py", line 380, in __iter__
    yield self  # This tells Task to wait for completion.
  File "/usr/lib/python3.5/asyncio/tasks.py", line 304, in _wakeup
    future.result()
  File "/usr/lib/python3.5/asyncio/futures.py", line 293, in result
    raise self._exception
aiorcon.exceptions.RCONClosedError: Connection closed before message was received
991jo commented 5 years ago

The first error is a timeout error. This means that the RCON connection ran into a timeout. For the second one I have to dig deeper into the code of the RCON library. In both cases something with your connection to the server is wrong. Are you sure that the servers are really running RCON and that your password is correct? What kind of servers (CSGO, TF2, L4D2, ...) are you monitoring?

larrryfisherthefisherman commented 5 years ago

Yes I am sure they are running RCON. They respond and then other times they throw out these errors.

They are all CSGO servers.

991jo commented 5 years ago

Okay, can you see a correlation between the times when this error occurs and what is going on on the servers? Maybe the server kills the connection or does not answer when it is changing the map?

larrryfisherthefisherman commented 5 years ago

Actually scratch all that it does seem to be related to map changes is there solution we can find so that this doesn't effect the graphs in Grafana?

991jo commented 5 years ago

You could tell your grafana to connect the missing points? If the connection fails it only returns the srcds_up 0 and no other metrics. This means that connecting the other metrics would smooth out your graphs.