search

bigcy / tungsten-replicator

Automatically exported from code.google.com/p/tungsten-replicator
0 stars 0 forks source link

Add a timeout in PrimaryKeyFilter in order to renew the connection if it has not been used for long #670

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
The connection is never renewed, so if a new table is checked for primary key 
filter, the filter will break if the connection was not used for a long period 
(around 20 days) with the following error :

INFO | jvm 1 | 2013/07/03 10:47:20 | 
com.continuent.tungsten.replicator.ReplicatorException: Filter failed 
processing primary key information 
INFO | jvm 1 | 2013/07/03 10:47:20 | at 
com.continuent.tungsten.enterprise.replicator.filter.PrimaryKeyFilter.filter(Pri
maryKeyFilter.java:181) 
INFO | jvm 1 | 2013/07/03 10:47:20 | at 
com.continuent.tungsten.replicator.pipeline.SingleThreadStageTask.runTask(Single
ThreadStageTask.java:279) 
INFO | jvm 1 | 2013/07/03 10:47:20 | at 
com.continuent.tungsten.replicator.pipeline.SingleThreadStageTask.run(SingleThre
adStageTask.java:120) 
INFO | jvm 1 | 2013/07/03 10:47:20 | at java.lang.Thread.run(Thread.java:619) 
INFO | jvm 1 | 2013/07/03 10:47:20 | Caused by: 
java.sql.SQLNonTransientConnectionException: Could not read resultset: 
Incomplete read! Expected 4, got -1 
INFO | jvm 1 | 2013/07/03 10:47:20 | at 
org.drizzle.jdbc.internal.SQLExceptionMapper.get(SQLExceptionMapper.java:97) 
INFO | jvm 1 | 2013/07/03 10:47:20 | at 
org.drizzle.jdbc.DrizzleStatement.executeQuery(DrizzleStatement.java:135) 
INFO | jvm 1 | 2013/07/03 10:47:20 | at 
org.drizzle.jdbc.MySQLDatabaseMetaData.getColumns(MySQLDatabaseMetaData.java:130
) 
INFO | jvm 1 | 2013/07/03 10:47:20 | at 
com.continuent.tungsten.replicator.database.MySQLDrizzleDatabase.getColumnsResul
tSet(MySQLDrizzleDatabase.java:56) 
INFO | jvm 1 | 2013/07/03 10:47:20 | at 
com.continuent.tungsten.replicator.database.AbstractDatabase.findTable(AbstractD
atabase.java:889) 
INFO | jvm 1 | 2013/07/03 10:47:20 | at 
com.continuent.tungsten.enterprise.replicator.filter.PrimaryKeyFilter.checkForPK
(PrimaryKeyFilter.java:296) 
INFO | jvm 1 | 2013/07/03 10:47:20 | at 
com.continuent.tungsten.enterprise.replicator.filter.PrimaryKeyFilter.filter(Pri
maryKeyFilter.java:177) 
INFO | jvm 1 | 2013/07/03 10:47:20 | ... 3 more 
INFO | jvm 1 | 2013/07/03 10:47:20 | Caused by: 
org.drizzle.jdbc.internal.common.QueryException: Could not read resultset: 
Incomplete read! Expected 4, got -1 
INFO | jvm 1 | 2013/07/03 10:47:20 | at 
org.drizzle.jdbc.internal.mysql.MySQLProtocol.executeQuery(MySQLProtocol.java:45
4) 
INFO | jvm 1 | 2013/07/03 10:47:20 | at 
org.drizzle.jdbc.DrizzleStatement.executeQuery(DrizzleStatement.java:131) 
INFO | jvm 1 | 2013/07/03 10:47:20 | ... 8 more 
INFO | jvm 1 | 2013/07/03 10:47:20 | Caused by: java.io.IOException: Incomplete 
read! Expected 4, got -1 
INFO | jvm 1 | 2013/07/03 10:47:20 | at 
org.drizzle.jdbc.internal.common.packet.RawPacket.readLengthSeq(RawPacket.java:9
7) 
INFO | jvm 1 | 2013/07/03 10:47:20 | at 
org.drizzle.jdbc.internal.common.packet.RawPacket.nextPacket(RawPacket.java:51) 
INFO | jvm 1 | 2013/07/03 10:47:20 | at 
org.drizzle.jdbc.internal.common.packet.SyncPacketFetcher.getRawPacket(SyncPacke
tFetcher.java:42) 
INFO | jvm 1 | 2013/07/03 10:47:20 | at 
org.drizzle.jdbc.internal.mysql.MySQLProtocol.executeQuery(MySQLProtocol.java:45
1) 
INFO | jvm 1 | 2013/07/03 10:47:20 | ... 9 more 

and the replicator goes into the error state. 

Putting the replicator back online fixes this issue.

Original issue reported on code.google.com by stephane...@continuent.com on 14 Aug 2013 at 3:17

GoogleCodeExporter commented 9 years ago 
This issue was closed by revision r1594.

Original comment by stephane...@continuent.com on 14 Aug 2013 at 3:20

GoogleCodeExporter commented 9 years ago 
This was committed.
There is no easy way to test it (besides waiting 25 days or so).

Original comment by stephane...@continuent.com on 14 Aug 2013 at 3:23

GoogleCodeExporter commented 9 years ago 
Need more info to test this matter. What timeouts are involved?

Original comment by g.maxia on 20 Aug 2013 at 1:13

GoogleCodeExporter commented 9 years ago

Original comment by linas.vi...@continuent.com on 26 Aug 2013 at 1:54

GoogleCodeExporter commented 9 years ago 
There won't be a 2.1.3.

Original comment by linas.vi...@continuent.com on 17 Sep 2013 at 10:13

GoogleCodeExporter commented 9 years ago 
Looking at the code, this should do what it is needed.
I am worried about the timeout being altered behind the user's back, but since 
it has been happening for years without issues, I guess we can live with that, 
at least for now.

Original comment by g.maxia on 18 Dec 2013 at 4:42

GoogleCodeExporter commented 9 years ago 
An entry has been added to the 2.2.0 release notes: 

The PrimaryKeyFilter would not renew connections to the master to determine the 
primary key information. When replication had been running for a long time, the 
active connection would be dropped, but never renewed. The filter has been 
updated to re-connect on failure.

Original comment by mc.br...@continuent.com on 18 Dec 2013 at 4:48