Fling-Asia / tonydbc

High-level DB connector for MariaDB
https://github/Fling-Asia/tonydbc
MIT License
1 stars 0 forks source link

infile permission on reconnect #7

Open MichaelCurrie opened 3 days ago

MichaelCurrie commented 3 days ago

This repo has no issue tracker available so I will record this here:

https://github.com/mariadb-corporation/mariadb-connector-python

We have identified an issue with the mariadb connector for python: even if a connection is made with local_infile=True:

                self._mariatonydbcn = mariadb.connect(
                    host=self.host,
                    user=self.user,
                    password=self.password,
                    database=self.database,
                    client_flag=MULTI_STATEMENTS,
                    autocommit=self.autocommit,
                    # connect_timeout=30,  # Default is 0, in seconds (have not yet tried this)
                    read_timeout=3600,  # 15,  # in seconds
                    write_timeout=3600,  # 20  # in seconds
                    local_infile=True,
                    compress=True,
                )
                # ...
                self._mariatonydbcn.auto_reconnect = True

if auto_reconnect = True, then on reconnect, it will reset the local_infile to False.

Minimal example of reproducing the bad behaviour:

import code
import pandas as pd
from tonydbc import load_dotenvs, TonyDBC, set_MYSQL_DATABASE

load_dotenvs()
set_MYSQL_DATABASE()

mock_df = pd.DataFrame(
    [(200, 5, 13)] * 5,
    columns=["stock_check_id", "planned_num_segments", "machine_id"],
)

with TonyDBC() as m:
    m.append_to_table("batch", mock_df)

    # Force a reconnection: please run:
    # SHOW PROCESSLIST;
    # KILL <id>;
    # (where <id> is the correct one)
    pass
    code.interact(banner="tag1", local=locals())

    m.append_to_table("batch", mock_df)
MichaelCurrie commented 3 days ago

To fix this, luckily we can just set self._mariatonydbcn.auto_reconnect = False since we already have an elegant reconnection callback implemented so don't need to rely on the buggy one provided by mariadb's python connector.

Fixed with 8dea31c