search

nshttpd / mikrotik-exporter

prometheus mikrotik device(s) exporter
BSD 3-Clause "New" or "Revised" License
549 stars 153 forks source link

Feature request: stay connected so the logs aren't spammed #65

Open wolph opened 4 years ago

wolph commented 4 years ago

Right now the collector seems to login/logout continuously. Would it be possible to simply stay logged in so you don't get logs filled with connected/disconnected messages?

welbymcroberts commented 4 years ago

This is a bit more invovled than reusing the existing connection. I've done a small test with connection re-use, and its incredibly easy to break. In this case i've set it so the RouterOS API Client is created and 'stored' in the collector struct, re-used if it is not nil. If two requests are made to the collector at the same time (or whilst another Collect is in progress) it will cause the second request to 'hang', and no more requests will be successful to that device.

There are few ways I can think of solving this

  1. Create a 'queue' per device, where all requests are handled in FIFO, this also would require some way of tracking which queue entries are for which HTTP request
  2. Hold a lock whilst the connection is in use. How the lock is handled needs to be considered, and timeouts need to also be implmented for the commands via the API
  3. Implement the 'multiple command' functionality in the Mikrotik API
  4. Seperate out the Collect 'process' from the scrape - this isn't necisarily a great idea, as freshness of the collect needs to be considered

I'm leaning towards the third option my self.

wolph commented 4 years ago

Hmm, that's harder than I had hoped. I was somehow assuming that the requests would be fast enough that a simple threading.Semaphore would be enough to solve those issues.

Effectively:

semaphore = threading.Semaphore()

# Do something
if semaphore.acquire(timeout=10):
    # Execute the code
else:
    # Timeout, reconnect maybe?
nshttpd commented 4 years ago

It's probably something that should be implemented in the upstream Client API Library either via a long lived connection with a health checking mechanism and reconnect logic. I think even with the multiple commands call it would still get the connected/disconnected messages in the log file.

RickDB commented 4 years ago

While from a security perspective not great the fix for me was excluding the account topic:

image