Get a lot of "internal server error" recently

[certrik]

Please help. Cannot really pinpoint the exact problem.

[Terrtia]

Hi @certrik !

Do you have any error messages in the Flask output ? Do you know on which endpoints you get this Error ?

[certrik]

Hi @Terrtia screen -r Flask_AIL tells me

redis.exceptions.ConnectionError: Error 111 connecting to localhost:6382. Connection refused.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/data/AIL/ail-framework/AILENV/lib/python3.7/site-packages/redis/connection.py", line 484, in connect
    sock = self._connect()
  File "/data/AIL/ail-framework/AILENV/lib/python3.7/site-packages/redis/connection.py", line 541, in _connect
    raise err
  File "/data/AIL/ail-framework/AILENV/lib/python3.7/site-packages/redis/connection.py", line 529, in _connect
    sock.connect(socket_address)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/data/AIL/ail-framework/AILENV/lib/python3.7/site-packages/flask/app.py", line 2447, in wsgi_app
    response = self.full_dispatch_request()
  File "/data/AIL/ail-framework/AILENV/lib/python3.7/site-packages/flask/app.py", line 1952, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/data/AIL/ail-framework/AILENV/lib/python3.7/site-packages/flask/app.py", line 1821, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/data/AIL/ail-framework/AILENV/lib/python3.7/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/data/AIL/ail-framework/AILENV/lib/python3.7/site-packages/flask/app.py", line 1950, in full_dispatch_request
    rv = self.dispatch_request()
  File "/data/AIL/ail-framework/AILENV/lib/python3.7/site-packages/flask/app.py", line 1936, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/data/AIL/ail-framework/var/www/blueprints/root.py", line 60, in login
    user = User.get(username)
  File "/data/AIL/ail-framework/bin/packages/User.py", line 36, in get
    return self_class(id)
  File "/data/AIL/ail-framework/bin/packages/User.py", line 23, in __init__
    if self.r_serv_db.hexists('user:all', id):
  File "/data/AIL/ail-framework/AILENV/lib/python3.7/site-packages/redis/client.py", line 1959, in hexists
    return self.execute_command('HEXISTS', name, key)
  File "/data/AIL/ail-framework/AILENV/lib/python3.7/site-packages/redis/client.py", line 673, in execute_command
    connection.send_command(*args)
  File "/data/AIL/ail-framework/AILENV/lib/python3.7/site-packages/redis/connection.py", line 610, in send_command
    self.send_packed_command(self.pack_command(*args))
  File "/data/AIL/ail-framework/AILENV/lib/python3.7/site-packages/redis/connection.py", line 585, in send_packed_command
    self.connect()
  File "/data/AIL/ail-framework/AILENV/lib/python3.7/site-packages/redis/connection.py", line 489, in connect
    raise ConnectionError(self._error_message(e))
redis.exceptions.ConnectionError: Error 111 connecting to localhost:6382. Connection refused.
172.31.1.32 - - [24/Feb/2021 22:35:35] "POST /login HTTP/1.1" 500 -
172.31.1.32 - - [24/Feb/2021 22:35:35] "GET /favicon.ico HTTP/1.1" 302 -
172.31.1.32 - - [24/Feb/2021 22:35:36] "GET /login?next=%2Ffavicon.ico HTTP/1.1" 200 -

after trying to log in.

[certrik]

Hi @Terrtia , this happens a lot, I think if the load on AIL is high. The ressources (CPU, RAM, etc) on the server itself is in no way maxed out. Do you have any idea what happens there? If I restart AIL it works properly again.

[KhaledEid020]

Same issue

[KhaledEid020]

VT submission is disabled Error 111 connecting to localhost:6383. Connection refused. Module CreditCards initialized Traceback (most recent call last): File "/home/socsoc/ail-framework/AILENV/lib/python3.10/site-packages/redis/connection.py", line 264, in connect sock = self.retry.call_with_retry( File "/home/socsoc/ail-framework/AILENV/lib/python3.10/site-packages/redis/retry.py", line 46, in call_with_retry return do() File "/home/socsoc/ail-framework/AILENV/lib/python3.10/site-packages/redis/connection.py", line 265, in lambda: self._connect(), lambda error: self.disconnect(error) File "/home/socsoc/ail-framework/AILENV/lib/python3.10/site-packages/redis/connection.py", line 627, in _connect raise err File "/home/socsoc/ail-framework/AILENV/lib/python3.10/site-packages/redis/connection.py", line 615, in _connect sock.connect(socket_address) ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/socsoc/ail-framework/var/www/./Flask_server.py", line 41, in from blueprints.objects_item import objects_item File "/home/socsoc/ail-framework/var/www/blueprints/objects_item.py", line 29, in from lib import module_extractor File "/home/socsoc/ail-framework/bin/lib/module_extractor.py", line 45, in 'infoleak:automatic-detection="iban"': Iban(queue=False), File "/home/socsoc/ail-framework/bin/modules/Iban.py", line 52, in init self.redis_logger.info(f'Module {self.module_name} initialized') File "/home/socsoc/ail-framework/AILENV/lib/python3.10/site-packages/pubsublogger/publisher.py", line 76, in info log('INFO', message) File "/home/socsoc/ail-framework/AILENV/lib/python3.10/site-packages/pubsublogger/publisher.py", line 62, in log redis_instance.publish(c, message) File "/home/socsoc/ail-framework/AILENV/lib/python3.10/site-packages/redis/commands/core.py", line 5191, in publish return self.execute_command("PUBLISH", channel, message, kwargs) File "/home/socsoc/ail-framework/AILENV/lib/python3.10/site-packages/redis/client.py", line 533, in execute_command conn = self.connection or pool.get_connection(command_name, options) File "/home/socsoc/ail-framework/AILENV/lib/python3.10/site-packages/redis/connection.py", line 1086, in get_connection connection.connect() File "/home/socsoc/ail-framework/AILENV/lib/python3.10/site-packages/redis/connection.py", line 270, in connect raise ConnectionError(self._error_message(e)) redis.exceptions.ConnectionError: Error 111 connecting to localhost:6380. Connection refused.