adrgs / requestrepo

Analyze HTTP and DNS requests and create custom DNS records for your subdomain
https://requestrepo.com
MIT License
372 stars 19 forks source link

Redis: max number of clients reached #35

Closed adrgs closed 8 months ago

adrgs commented 8 months ago

The DNS service will go down if there are too many redis clients connected at once

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/socketserver.py", line 691, in process_request_thread
    self.finish_request(request, client_address)
  File "/usr/local/lib/python3.11/socketserver.py", line 361, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "/usr/local/lib/python3.11/socketserver.py", line 755, in __init__
    self.handle()
  File "/usr/local/lib/python3.11/site-packages/dnslib/server.py", line 165, in handle
    rdata = self.get_reply(data)
            ^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/dnslib/server.py", line 182, in get_reply
    reply = resolver.resolve(request,self)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/./ns.py", line 73, in resolve
    new_record = self.resolve_cname(reply)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/./ns.py", line 27, in resolve_cname
    data = get_dns_record(str(reply.q.qname), "CNAME")
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/./ns.py", line 158, in get_dns_record
    result = r.get(f"dns:{dtype}:{domain}")
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/redis/commands/core.py", line 1829, in get
    return self.execute_command("GET", name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/redis/client.py", line 533, in execute_command
    conn = self.connection or pool.get_connection(command_name, **options)
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/redis/connection.py", line 1086, in get_connection
    connection.connect()
  File "/usr/local/lib/python3.11/site-packages/redis/connection.py", line 276, in connect
    self.on_connect()
  File "/usr/local/lib/python3.11/site-packages/redis/connection.py", line 379, in on_connect
    self.read_response()
  File "/usr/local/lib/python3.11/site-packages/redis/connection.py", line 500, in read_response
    response = self._parser.read_response(disable_decoding=disable_decoding)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/redis/_parsers/hiredis.py", line 133, in read_response
    raise response
redis.exceptions.ConnectionError: max number of clients reached
----------------------------------------

Looks like connections are not properly closed, will need to investigate.

adrgs commented 8 months ago

Implemented connection pools in #36 , will see how it performs and reopen the issue in case it doesn't work.