zxs / tungsten-replicator

Automatically exported from code.google.com/p/tungsten-replicator
0 stars 0 forks source link

Fixmysqlstrings.js filter does not correctly translate binary with trailing x'00' characters #1113

Open GoogleCodeExporter opened 9 years ago

GoogleCodeExporter commented 9 years ago
What steps will reproduce the problem?

1. Set up master/slave replication with following filters enabled on extract:  
colnames,fixmysqlstrings,pkey.  (This si a common data warehouse extraction 
pattern.)

2. Execute the following query on the master. 

use test
set session binlog_format=row;
drop table if exists uuid_data;
create table uuid_data(id int primary key, d char(10), uuid binary(16));
insert into uuid_data values(1, 'hi', x'DD9D7EFFFBD247CAB3EF6167B0C4AD01');
insert into uuid_data values(2, 'bye', x'DD9D7EFFFBD247CAB3EF6167B0C4AD00');
insert into uuid_data values(3, 'bye', x'DD9D7EFFFBD247CAB3EF6167B0C4AD10');
insert into uuid_data values(4, 'bye', x'DD9D7EFFFBD247CAB300000000000000');
select id,d,hex(uuid) from uuid_data;

3. Check the values inserted into the THL for each transaction. 

What is the expected output?

THL should show values like the following: 

SEQ# = 112827 / FRAG# = 0 (last frag)
- TIME = 2015-04-09 11:40:12.0
- EPOCH# = 0
- EVENTID = mysql-bin.017264:0000000041379007;-1
- SOURCEID = logos1
- METADATA = 
[mysql_server_id=1;dbms_type=mysql;service=batch;shard=test;strings=utf8]
- TYPE = com.continuent.tungsten.replicator.event.ReplDBMSEvent
- OPTIONS = [foreign_key_checks = 1, unique_checks = 1]
- SQL(0) =
 - ACTION = INSERT
 - SCHEMA = test
 - TABLE = uuid_data
 - ROW# = 0
  - COL(1: id) = 2
  - COL(2: d) = bye
  - COL(3: uuid) = DD9D7EFFFBD247CAB3EF6167B0C4AD00
  - KEY(1: id) = NULL

What do you see instead?

Trailing zeros on the uuid field are truncated. 

What is the possible cause?

The operation to convert to hex strings uses 
javax.xml.bind.DatatypeConverter.printHexBinary() does not handle trailing null 
bytes. 

What is the proposed solution?

Look up column metadata and pad the value out to the expected size for BINARY 
but not VARBINARY.

Additional information

Fixed in maintenance 3.0 branch and trunk. 
...

Use labels and text to provide additional information.

Original issue reported on code.google.com by robert.h...@continuent.com on 9 Apr 2015 at 6:45

GoogleCodeExporter commented 9 years ago
This issue was updated by revision r2810.

Added support for padding BINARY column values to add trailing null characters 
when converting to hex. 

Original comment by robert.h...@continuent.com on 9 Apr 2015 at 6:48

GoogleCodeExporter commented 9 years ago
This issue was updated by revision r2811.

Removed comment accidentally left in from previous fix. 

Original comment by robert.h...@continuent.com on 10 Apr 2015 at 10:26