zhongbiaodev / py-mysql-elasticsearch-sync

MySQL to Elasticsearch sync tool, written in Python.
MIT License
178 stars 54 forks source link

执行到最后报错:'NoneType' object is not subscriptable` #13

Closed anuxs closed 2 months ago

anuxs commented 7 years ago

Traceback (most recent call last): File "/usr/local/bin/es-sync", line 11, in sys.exit(start()) File "/usr/local/lib/python3.5/site-packages/es_sync/init.py", line 507, in start instance.run() File "/usr/local/lib/python3.5/site-packages/es_sync/init.py", line 497, in run self._sync_from_binlog() File "/usr/local/lib/python3.5/site-packages/es_sync/init.py", line 483, in _sync_from_binlog self._updater(docs) File "/usr/local/lib/python3.5/site-packages/es_sync/init.py", line 171, in _updater for item in data: File "/usr/local/lib/python3.5/site-packages/es_sync/init.py", line 197, in _processor for item in data: File "/usr/local/lib/python3.5/site-packages/es_sync/init.py", line 228, in _mapper for item in data: File "/usr/local/lib/python3.5/site-packages/es_sync/init.py", line 278, in _binlog_loader for binlogevent in stream: File "/usr/local/lib/python3.5/site-packages/pymysqlreplication/binlogstream.py", line 361, in fetchone self.connect_to_stream() File "/usr/local/lib/python3.5/site-packages/pymysqlreplication/binlogstream.py", line 269, in connect_to_stream self.log_file, self.log_pos = cur.fetchone()[:2] TypeError: 'NoneType' object is not subscriptable

数据已经导入成功了,可以检索到。最后的错误是什么?

anuxs commented 7 years ago

原因大致猜测如下: 没有启用mysql的binlog模式。第一阶段是用mysqldump作全部导入,第二阶段是检测binlog做增量导入。 但是程序没有考虑binlog没有启用、导入不成功的情况。

因为我mac电脑一直启动binlong不成功,所以执行到第二阶段报错。

另外一个go写的的同类型的工具一开始就报错说binlog没有启用。

麻烦作者处理下吧。问题提了几个月了。

xiaoxiaojuan commented 7 years ago

你好,我在centos下开启了binlog后,再灌数据,就一直是执行状态,不动了,看日志出现Sync binlog_file: mysql-bin.000008 ,很久都不结束,不知道这是怎么了

xiaoxiaojuan commented 7 years ago

1.开启binlog后无法实现同步,show master status; 发现日志是mysql-bin.000008,但是同步工具中的是mysql-bin.000008 pos:4 ,通过flush logs 到mysql-bin.000008,然后把binlog.info中的改成一样的pos,还是不行。不知道什么问题导致无法用binlog同步,只好关闭binlog 2.关掉binlog后还是出现错误pymysql.err.InternalError: (1236, u'Binary log is not open') 后来看代码知道把配置的binlog.info 删掉就行了,会写入数据,但是无法同步 凑合能用

anuxs commented 7 years ago

mysql-bin.000008 一直停在那里了。不增量了,没动静。