search

funilrys / PyFunceble

The tool to check the availability or syntax of domain, IP or URL.
https://pyfunceble.github.io
Apache License 2.0
289 stars 44 forks source link

BUG: sqlalche braekes after finishing... #346

Closed spirillen closed 6 months ago

spirillen commented 6 months ago

Description

After the test have "completed" sqlalchemy complains about broken mysql connection

Configuration

cli_decoding:
    adblock: false
    aggressive: false
    rpz: false
    wildcard: false
cli_testing:
    autocontinue: false
    chancy_tester: false
    ci:
        active: false
        branch: master
        command: null
        commit_message: PyFunceble - AutoSave
        distribution_branch: master
        end_command: null
        end_commit_message: PyFunceble - Results
        max_exec_minutes: 15
    cidr_expand: false
    complements: false
    cooldown_time: 0.0
    days_between:
        db_clean: 28
        db_retest: 1
    db_type: csv
    display_mode:
        all: false
        colour: true
        dots: false
        execution_time: false
        less: true
        max_registrar: 15
        percentage: true
        quiet: false
        registrar: false
        simple: false
        status: all
    file_filter: null
    file_generation:
        analytic: true
        hosts: false
        merge_output_dirs: false
        no_file: false
        plain: true
        unified_results: false
    hosts_ip: 0.0.0.0
    inactive_db: true
    local_network: false
    max_workers: null
    mining: false
    preload_file: false
    sorting_mode:
        hierarchical: false
        standard: true
    testing_mode:
        availability: true
        reputation: false
        syntax: false
    whois_db: true
collection:
    preferred_status_origin: recommended
    push: false
    url_base: https://collection.dead-hosts.funilrys.com
debug:
    active: false
    level: info
dns:
    delay: 0.0
    follow_server_order: true
    protocol: UDP
    server: null
    trust_server: false
http_codes:
    list:
        potentially_down:
        - 400
        - 402
        - 404
        - 409
        - 410
        - 412
        - 414
        - 415
        - 416
        - 451
        potentially_up:
        - 0
        - 300
        - 301
        - 302
        - 303
        - 304
        - 305
        - 307
        - 308
        - 403
        - 405
        - 406
        - 407
        - 408
        - 411
        - 413
        - 417
        - 418
        - 421
        - 422
        - 423
        - 424
        - 426
        - 428
        - 431
        - 500
        - 501
        - 502
        - 503
        - 504
        - 505
        - 506
        - 507
        - 508
        - 510
        - 511
        up:
        - 100
        - 101
        - 102
        - 200
        - 201
        - 202
        - 203
        - 204
        - 205
        - 206
        - 207
        - 208
        - 226
        - 429
    self_managed: false
links:
    api_date_format: https://pyfunceble.funilrys.com/api/date-format
    api_no_referrer: https://pyfunceble.funilrys.com/api/no-referrer
lookup:
    collection: false
    dns: true
    http_status_code: true
    netinfo: true
    reputation: false
    special: true
    timeout: 5
    whois: true
max_http_retries: 3
proxy:
    global:
        http: null
        https: null
    rules: []
share_logs: false
user_agent:
    browser: chrome
    custom: null
    platform: linux
verify_ssl_certificate: false

Reproduction

  1. pyfunceble -f matrix/source/porn_filters/explicit_content/wildcard.list

Expected behavior

No errors

Screenshots

image

Versions

OS: Ubuntu 22.04.3

Python Version: 3.11

PyFunceble Version: pyfunceble 4.2.4.dev (Blue Duckling: Ixora)

Additional context

Fatal Error: (pymysql.err.OperationalError) (2006, "MySQL server has gone away (BrokenPipeError(32, 'Broken pipe'))")
[SQL: DELETE FROM pyfunceble_continue WHERE pyfunceble_continue.session_id = %(session_id_1)s]
[parameters: {'session_id_1': '9d6747bf4ab0fa349c99f4ff'}]
(Background on this error at: https://sqlalche.me/e/20/e3q8)
Traceback (most recent call last):
File "/home/$USER/.local/lib/python3.11/site-packages/pymysql/connections.py", line 803, in _write_bytes
self._sock.sendall(data)
BrokenPipeError: [Errno 32] Broken pipe

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/$USER/.local/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1965, in _exec_single_context
self.dialect.do_execute(
File "/home/$USER/.local/lib/python3.11/site-packages/sqlalchemy/engine/default.py", line 921, in do_execute
cursor.execute(statement, parameters)
File "/home/$USER/.local/lib/python3.11/site-packages/pymysql/cursors.py", line 153, in execute
result = self._query(query)
^^^^^^^^^^^^^^^^^^
File "/home/$USER/.local/lib/python3.11/site-packages/pymysql/cursors.py", line 322, in _query
conn.query(q)
File "/home/$USER/.local/lib/python3.11/site-packages/pymysql/connections.py", line 557, in query
self._execute_command(COMMAND.COM_QUERY, sql)
File "/home/$USER/.local/lib/python3.11/site-packages/pymysql/connections.py", line 861, in _execute_command
self._write_bytes(packet)
File "/home/$USER/.local/lib/python3.11/site-packages/pymysql/connections.py", line 806, in _write_bytes
raise err.OperationalError(
pymysql.err.OperationalError: (2006, "MySQL server has gone away (BrokenPipeError(32, 'Broken pipe'))")

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/home/$USER/.local/lib/python3.11/site-packages/PyFunceble/cli/system/launcher.py", line 1060, in start
self.run_standard_end_instructions()
File "/home/$USER/.local/lib/python3.11/site-packages/PyFunceble/cli/system/launcher.py", line 924, in run_standard_end_instructions
self.remove_unwanted_files()
File "/home/$USER/.local/lib/python3.11/site-packages/PyFunceble/cli/system/launcher.py", line 901, in remove_unwanted_files
remove_continue_dataset(protocol)
File "/home/$USER/.local/lib/python3.11/site-packages/PyFunceble/cli/system/launcher.py", line 858, in remove_continue_dataset
self.continue_dataset.cleanup(  # pylint: disable=unexpected-keyword-arg
File "/home/$USER/.local/lib/python3.11/site-packages/PyFunceble/dataset/db_base.py", line 104, in wrapper
return func(self, *args, **kwargs)  # pylint: disable=not-callable
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/$USER/.local/lib/python3.11/site-packages/PyFunceble/dataset/autocontinue/sql.py", line 86, in cleanup
).delete(synchronize_session=False)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/$USER/.local/lib/python3.11/site-packages/sqlalchemy/orm/query.py", line 3176, in delete
result: CursorResult[Any] = self.session.execute(
^^^^^^^^^^^^^^^^^^^^^
File "/home/$USER/.local/lib/python3.11/site-packages/sqlalchemy/orm/session.py", line 2262, in execute
return self._execute_internal(
^^^^^^^^^^^^^^^^^^^^^^^
File "/home/$USER/.local/lib/python3.11/site-packages/sqlalchemy/orm/session.py", line 2144, in _execute_internal
result: Result[Any] = compile_state_cls.orm_execute_statement(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/$USER/.local/lib/python3.11/site-packages/sqlalchemy/orm/bulk_persistence.py", line 1946, in orm_execute_statement
return super().orm_execute_statement(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/$USER/.local/lib/python3.11/site-packages/sqlalchemy/orm/context.py", line 293, in orm_execute_statement
result = conn.execute(
^^^^^^^^^^^^^
File "/home/$USER/.local/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1412, in execute
return meth(
^^^^^
File "/home/$USER/.local/lib/python3.11/site-packages/sqlalchemy/sql/elements.py", line 516, in _execute_on_connection
return connection._execute_clauseelement(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/$USER/.local/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1635, in _execute_clauseelement
ret = self._execute_context(
^^^^^^^^^^^^^^^^^^^^^^
File "/home/$USER/.local/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1844, in _execute_context
return self._exec_single_context(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/$USER/.local/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1984, in _exec_single_context
self._handle_dbapi_exception(
File "/home/$USER/.local/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 2339, in _handle_dbapi_exception
raise sqlalchemy_exception.with_traceback(exc_info[2]) from e
File "/home/$USER/.local/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1965, in _exec_single_context
self.dialect.do_execute(
File "/home/$USER/.local/lib/python3.11/site-packages/sqlalchemy/engine/default.py", line 921, in do_execute
cursor.execute(statement, parameters)
File "/home/$USER/.local/lib/python3.11/site-packages/pymysql/cursors.py", line 153, in execute
result = self._query(query)
^^^^^^^^^^^^^^^^^^
File "/home/$USER/.local/lib/python3.11/site-packages/pymysql/cursors.py", line 322, in _query
conn.query(q)
File "/home/$USER/.local/lib/python3.11/site-packages/pymysql/connections.py", line 557, in query
self._execute_command(COMMAND.COM_QUERY, sql)
File "/home/$USER/.local/lib/python3.11/site-packages/pymysql/connections.py", line 861, in _execute_command
self._write_bytes(packet)
File "/home/$USER/.local/lib/python3.11/site-packages/pymysql/connections.py", line 806, in _write_bytes
raise err.OperationalError(
sqlalchemy.exc.OperationalError: (pymysql.err.OperationalError) (2006, "MySQL server has gone away (BrokenPipeError(32, 'Broken pipe'))")
[SQL: DELETE FROM pyfunceble_continue WHERE pyfunceble_continue.session_id = %(session_id_1)s]
[parameters: {'session_id_1': '9d6747bf4ab0fa349c99f4ff'}]
(Background on this error at: https://sqlalche.me/e/20/e3q8)