julien-duponchelle / python-mysql-replication

Pure Python Implementation of MySQL replication protocol build on top of PyMYSQL
2.31k stars 678 forks source link

Fail when the ecoding is ucs2 #159

Open guerremdq opened 8 years ago

guerremdq commented 8 years ago

Hello,

I had a field in the table with charset ucs2

test_field  varchar(100) CHARACTER SET ucs2 DEFAULT NULL COMMENT

and the is failing with this error:

Traceback (most recent call last):
 File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner
   self.run()
 File "lazarus.py", line 85, in run
   for row in binlogevent.rows:
 File "/usr/local/lib/python2.7/dist-packages/pymysqlreplication/row_event.py", line 397, in rows
   self._fetch_rows()
 File "/usr/local/lib/python2.7/dist-packages/pymysqlreplication/row_event.py", line 392, in _fetch_rows
   self.__rows.append(self._fetch_one_row())
 File "/usr/local/lib/python2.7/dist-packages/pymysqlreplication/row_event.py", line 445, in _fetch_one_row
   row["values"] = self._read_column_data(self.columns_present_bitmap)
 File "/usr/local/lib/python2.7/dist-packages/pymysqlreplication/row_event.py", line 118, in _read_column_data
   values[name] = self.__read_string(1, column)
 File "/usr/local/lib/python2.7/dist-packages/pymysqlreplication/row_event.py", line 197, in __read_string
   string = string.decode(column.character_set_name)
LookupError: unknown encoding: ucs2

It's possible to support this encoding?

Thaks

baloo commented 8 years ago

I'd say it's related to #146. I would solve both problems by not trying to parse content and just return a byte array to user. And let user parse it's data.

This is a breaking change, but it's the safest we can try here.

Any thoughts?

julien-duponchelle commented 8 years ago

Sadly I agree with you. We will need to provide a sample for user to get the character_set_name if people want to do it like today.