search

epermana / tungsten-replicator

Automatically exported from code.google.com/p/tungsten-replicator
1 stars 0 forks source link

THL generated empty logs #956

Open GoogleCodeExporter opened 9 years ago

GoogleCodeExporter commented 9 years ago 
What steps will reproduce the problem?

1. Not sure, How to re produce them, Its just happened over night.
2.
3.

What is the expected output?

... Should contain sequence of statements in thl logs

What do you see instead?

... Empty log with size 0 bytes.

What version of the product are you using?

... 2.2.1

On what operating system?

...Ubuntu 12.04 LTS

Please provide any additional information below.

Notes:

... I have a tungsten setup with 3 mysql servers in multi master. Replication 
seems to working amazing, but last night it generated empty logs and broke 
services with each other. 

-rw-r----- 1 root adm 0 Jun 22 06:40 thl.data.0000001590
-rw-r----- 1 root adm 0 Jun 22 06:40 thl.data.0000001591
-rw-r----- 1 root adm 0 Jun 22 06:40 thl.data.0000001592
-rw-r----- 1 root adm 0 Jun 22 06:40 thl.data.0000001593
-rw-r----- 1 root adm 0 Jun 22 06:40 thl.data.0000001594
-rw-r----- 1 root adm 0 Jun 22 06:40 thl.data.0000001595
-rw-r----- 1 root adm 0 Jun 22 06:40 thl.data.0000001596
-rw-r----- 1 root adm 0 Jun 22 06:40 thl.data.0000001597
-rw-r----- 1 root adm 0 Jun 22 06:40 thl.data.0000001598
-rw-r----- 1 root adm 0 Jun 22 06:40 thl.data.0000001599
-rw-r----- 1 root adm 0 Jun 22 06:40 thl.data.0000001600
-rw-r----- 1 root adm 0 Jun 22 06:40 thl.data.0000001601
-rw-r----- 1 root adm 0 Jun 22 06:40 thl.data.0000001602
-rw-r----- 1 root adm 0 Jun 22 06:40 thl.data.0000001603
-rw-r----- 1 root adm 0 Jun 22 06:40 thl.data.0000001604
-rw-r----- 1 root adm 0 Jun 22 06:40 thl.data.0000001605
-rw-r----- 1 root adm 0 Jun 22 06:40 thl.data.0000001606
-rw-r----- 1 root adm 0 Jun 22 06:40 thl.data.0000001607
-rw-r----- 1 root adm 0 Jun 22 06:40 thl.data.0000001608
-rw-r----- 1 root adm 0 Jun 22 06:40 thl.data.0000001609
-rw-r----- 1 root adm 0 Jun 22 06:40 thl.data.0000001610
-rw-r----- 1 root adm 0 Jun 22 06:40 thl.data.0000001611
-rw-r----- 1 root adm 0 Jun 22 06:40 thl.data.0000001612
-rw-r----- 1 root adm 0 Jun 22 06:40 thl.data.0000001613
-rw-r----- 1 root adm 0 Jun 22 06:40 thl.data.0000001614
-rw-r----- 1 root adm 0 Jun 22 06:40 thl.data.0000001615
-rw-r----- 1 root adm 0 Jun 22 06:40 thl.data.0000001616
-rw-r----- 1 root adm 0 Jun 22 06:40 thl.data.0000001617
-rw-r----- 1 root adm 0 Jun 22 06:40 thl.data.0000001618
-rw-r----- 1 root adm 0 Jun 22 06:40 thl.data.0000001619
-rw-r----- 1 root adm 0 Jun 22 06:40 thl.data.0000001620
-rw-r----- 1 root adm 0 Jun 22 06:40 thl.data.0000001621
-rw-r----- 1 root adm 0 Jun 22 06:40 thl.data.0000001622
-rw-r----- 1 root adm 0 Jun 22 06:40 thl.data.0000001623
-rw-r----- 1 root adm 0 Jun 22 06:40 thl.data.0000001624
-rw-r----- 1 root adm 0 Jun 22 06:40 thl.data.0000001625

These empty logs are created from thl.data.0000001060 to thl.data.0000001625. 
How can i resolve this issue? Is this bug?

Original issue reported on code.google.com by swaroopk...@gmail.com on 24 Jun 2014 at 9:22

GoogleCodeExporter commented 9 years ago 
More information:

:~# trepctl -service db1 status
Processing status command...
NAME                     VALUE
----                     -----
appliedLastEventId     : NONE
appliedLastSeqno       : -1
appliedLatency         : -1.0
autoRecoveryEnabled    : true
autoRecoveryTotal      : 0
channels               : -1
clusterName            : db1
currentEventId         : NONE
currentTimeMillis      : 1403604232856
dataServerHost         : db1
extensions             : 
host                   : db1
latestEpochNumber      : -1
masterConnectUri       : thl://localhost:/
masterListenUri        : thl://db1:2112/
maximumStoredSeqNo     : -1
minimumStoredSeqNo     : -1
offlineRequests        : NONE
pendingError           : Replicator unable to go online due to error
pendingErrorCode       : NONE
pendingErrorEventId    : NONE
pendingErrorSeqno      : -1
pendingExceptionMessage: Unable to prepare plugin: class 
name=com.continuent.tungsten.replicator.thl.THL message=[Found invalid log file 
header; log must be purged up to this file to open: 
/opt/installs/cookbook/thl/db1/thl.data.0000001200]
pipelineSource         : UNKNOWN
relativeLatency        : -1.0
resourcePrecedence     : 99
rmiPort                : 10000
role                   : master
seqnoType              : java.lang.Long
serviceName            : db1
serviceType            : unknown
simpleServiceName      : db1
siteName               : default
sourceId               : db1
state                  : OFFLINE:ERROR
timeInStateSeconds     : 1447.389
transitioningTo        : 
uptimeSeconds          : 1447.86
useSSLConnection       : false
version                : Tungsten Replicator 2.2.1 build 403
Finished status command...

Original comment by swaroopk...@gmail.com on 24 Jun 2014 at 10:04

GoogleCodeExporter commented 9 years ago 
Have you checked disk space?

Original comment by linas.vi...@continuent.com on 25 Jun 2014 at 11:36

GoogleCodeExporter commented 9 years ago 
Yes, I had around 600GB left on each server, I discarded empty logs and 
processed tungsten. Will this leads to any data loss?

Original comment by swaroopk...@gmail.com on 25 Jun 2014 at 11:42