Altinity / clickhouse-mysql-data-reader

utility to read mysql data
MIT License
368 stars 94 forks source link

map argument #1 must support iteration error on replication #141

Closed MaxFedotov closed 5 years ago

MaxFedotov commented 5 years ago

Hello, we are evaluating clickhouse-mysql-data-reader replication, and got following error in logs:

2018-12-04 16:54:15,973/1543942455.973548:DEBUG:class:<class 'clickhouse_mysql.writer.processwriter.ProcessWriter'> process()
2018-12-04 16:54:15,974/1543942455.974890:INFO:CSVWriter() csv_file_path=None csv_file_path_prefix=/root/ch_ csv_file_path_suffix_parts=[] csv_keep_file=False dst_schema=sl dst_table=None
2018-12-04 16:54:15,975/1543942455.975752:INFO:CSVWriter() self.path=/root/ch_1543942455.9753294_c9b08f22-72ec-48f8-8b0a-f82e614de73c.csv
2018-12-04 16:54:15,976/1543942455.976219:WARNING:No events to insert. class: <class 'clickhouse_mysql.writer.csvwriter.CSVWriter'>
2018-12-04 16:54:15,976/1543942455.976954:INFO:CHCSWriter() connection_settings={'host': '127.0.0.1', 'port': 9000, 'user': 'default', 'password': ''} dst_schema=sl dst_table=None
2018-12-04 16:54:15,977/1543942455.977393:DEBUG:class:<class 'clickhouse_mysql.writer.chcsvwriter.CHCSVWriter'> insert 1 rows
--this line was added by me trying to figure out what was going on
2018-12-04 16:54:15,977/1543942455.977737:DEBUG:event: <clickhouse_mysql.event.event.Event object at 0x0000000092b8afe0>, attrs: {'schema': 'sl', 'table': None, 'filename': '/root/ch_1543942455.9753294_c9b08f22-72ec-48f8-8b0a-f82e614de73c.csv', 'fieldnames': None}, events: [<clickhouse_mysql.event.event.Event object at 0x0000000092b8afe0>]
Process Process-11:
Traceback (most recent call last):
  File "/root/pypi/pypy3.5-6.0.0-linux_x86_64-portable/lib-python/3/multiprocessing/process.py", line 249, in _bootstrap
    self.run()
  File "/root/pypi/pypy3.5-6.0.0-linux_x86_64-portable/lib-python/3/multiprocessing/process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "/root/pypi/pypy3.5-6.0.0-linux_x86_64-portable/site-packages/clickhouse_mysql-0.0.20180321-py3.5.egg/clickhouse_mysql/writer/processwriter.py", line 34, in process
    writer.push()
  File "/root/pypi/pypy3.5-6.0.0-linux_x86_64-portable/site-packages/clickhouse_mysql-0.0.20180321-py3.5.egg/clickhouse_mysql/writer/csvwriter.py", line 145, in push
    self.next_writer_builder.get().insert(event)
  File "/root/pypi/pypy3.5-6.0.0-linux_x86_64-portable/site-packages/clickhouse_mysql-0.0.20180321-py3.5.egg/clickhouse_mysql/writer/chcsvwriter.py", line 63, in insert
    ', '.join(map(lambda column: '`%s`' % column, event.fieldnames)),
TypeError: map argument #1 must support iteration
2018-12-04 16:54:16,530/1543942456.530018:DEBUG:class:<class 'clickhouse_mysql.writer.processwriter.ProcessWriter'> process() done

The thing is, that file /root/ch_1543942455.9753294_c9b08f22-72ec-48f8-8b0a-f82e614de73c.csv doesn't exists at all.

sunsingerus commented 5 years ago

fixed