julien-duponchelle / python-mysql-replication

Pure Python Implementation of MySQL replication protocol build on top of PyMYSQL
2.33k stars 679 forks source link

max_allowed_packet issue when using ## for binlogEvent in stream #301

Open BlackWatchuser opened 5 years ago

BlackWatchuser commented 5 years ago

error message like below:

log event entry exceeded max_allowed_packet; Increase max_allowed_packet on master

the stack is:

  for binlog_event in stream:
  File "/usr/lib/python2.7/dist-packages/pymysqlreplication/binlogstream.py", line 408, in fetchone
    pkt = self._stream_connection._read_packet()
  File "/usr/local/lib/python2.7/site-packages/PyMySQL-0.7.11-py2.7.egg/pymysql/connections.py", line 1014, in _read_packet
    packet.check_error()
  File "/usr/local/lib/python2.7/site-packages/PyMySQL-0.7.11-py2.7.egg/pymysql/connections.py", line 393, in check_error
    err.raise_mysql_exception(self._data)
  File "/usr/local/lib/python2.7/site-packages/PyMySQL-0.7.11-py2.7.egg/pymysql/err.py", line 107, in raise_mysql_exception
    raise errorclass(errno, errval)
InternalError: (1236, u"log event entry exceeded max_allowed_packet; Increase max_allowed_packet on master;

I checked master's max_allowed_packet setting, and i found it was setting to 1GB, so i adding the setting on my code, here's the detail:

conn_setting = {'host': HOST, 'port': 3306, 'user': USER, 'passwd': PASSWORD, 'charset': 'utf8', 'max_allowed_packet':1*1024*1024*1024}
        binlog2sql = Binlog2sql(connection_settings=conn_setting, start_file=FILE, start_pos=POS,
                                end_file="", end_pos="", start_time="",
                                stop_time="", only_schemas="", only_tables="",
                                no_pk=False, flashback=False, stop_never=False,
                                back_interval=1.0, only_dml=False, sql_type=['INSERT', 'UPDATE', 'DELETE'])

but it seems like having no effect. please check this out.

dongwook-chan commented 2 years ago

@BlackWatchuser Could you please provide a scenario to reproduce this exception?