search

epermana / tungsten-replicator

Automatically exported from code.google.com/p/tungsten-replicator
1 stars 0 forks source link

Unable to start a slave (ORA-06550: line , column :) #942

Open GoogleCodeExporter opened 9 years ago

GoogleCodeExporter commented 9 years ago 
What steps will reproduce the problem?

1. Set up a clean database with an empty set of schemas
2. Install an oracle to oracle cluster. Do not bring the master online. 
3. Ensure setupCDC.sh finished successfully on the slave. 
4. Try to start the slave

What is the expected output?

The slave should start. 

What do you see instead?

Processing status command...
NAME                     VALUE
----                     -----
appliedLastEventId     : NONE
appliedLastSeqno       : -1
appliedLatency         : -1.0
autoRecoveryEnabled    : false
autoRecoveryTotal      : 0
channels               : -1
clusterName            : shoe_master
currentEventId         : NONE
currentTimeMillis      : 1402697694659
dataServerHost         : odb01.dev1.rsp.shoebuy.com
extensions             : 
host                   : odb01.dev1.rsp.shoebuy.com
latestEpochNumber      : -1
masterConnectUri       : thl://tungsten-master:2112/
masterListenUri        : thl://tungsten-slave:2112/
maximumStoredSeqNo     : -1
minimumStoredSeqNo     : -1
offlineRequests        : NONE
pendingError           : Replicator unable to go online due to error
pendingErrorCode       : NONE
pendingErrorEventId    : NONE
pendingErrorSeqno      : -1
pendingExceptionMessage: Unable to prepare plugin: class 
name=com.continuent.tungsten.replicator.thl.THL message=[java.sql.SQLException: 
ORA-06550: line , column :
                         ORA-06512: at "SYS.DBMS_CDC_PUBLISH", line 999
                         ORA-06512: at line 1
                         ]
pipelineSource         : UNKNOWN
relativeLatency        : -1.0
resourcePrecedence     : 99
rmiPort                : 10003
role                   : slave
seqnoType              : java.lang.Long
serviceName            : shoe_master
serviceType            : unknown
simpleServiceName      : shoe_master
siteName               : default
sourceId               : odb01.dev1.rsp.shoebuy.com
state                  : OFFLINE:ERROR
timeInStateSeconds     : 37.893
transitioningTo        : 
uptimeSeconds          : 49.722
useSSLConnection       : false
version                : Tungsten Replicator 2.2.1 build 403
Finished status command...

What version of the product are you using?

2.2.1 build 403

On what operating system?

CentOS 6.5

Please provide any additional information below.

Stack trace from log:
INFO   | jvm 1    | 2014/06/13 18:14:16 | 2014-06-13 18:14:16,770 [ - 
WrapperSimpleAppMain] ERROR management.OpenReplicatorManager Online operation 
failed
INFO   | jvm 1    | 2014/06/13 18:14:16 | 
com.continuent.tungsten.replicator.management.ReplicatorStateException: State 
transition failed causing emergency recovery: state=OFFLINE:NORMAL 
transition=OFFLINE-GO-ONLINE-1 event=GoOnlineEvent
INFO   | jvm 1    | 2014/06/13 18:14:16 |       at 
com.continuent.tungsten.replicator.management.OpenReplicatorManager.getStateMach
ineException(OpenReplicatorManager.java:593)
INFO   | jvm 1    | 2014/06/13 18:14:16 |       at 
com.continuent.tungsten.replicator.management.OpenReplicatorManager.onCompletion
(OpenReplicatorManager.java:559)
INFO   | jvm 1    | 2014/06/13 18:14:16 |       at 
com.continuent.tungsten.fsm.event.EventProcessor.run(EventProcessor.java:108)
INFO   | jvm 1    | 2014/06/13 18:14:16 |       at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
INFO   | jvm 1    | 2014/06/13 18:14:16 |       at 
java.util.concurrent.FutureTask.run(FutureTask.java:262)
INFO   | jvm 1    | 2014/06/13 18:14:16 |       at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
INFO   | jvm 1    | 2014/06/13 18:14:16 |       at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
INFO   | jvm 1    | 2014/06/13 18:14:16 | Caused by: 
com.continuent.tungsten.fsm.core.TransitionFailureException: Replicator unable 
to go online due to error
INFO   | jvm 1    | 2014/06/13 18:14:16 |       at 
com.continuent.tungsten.replicator.management.OpenReplicatorManager$OfflineToSyn
chronizingAction.doAction(OpenReplicatorManager.java:1385)
INFO   | jvm 1    | 2014/06/13 18:14:16 |       at 
com.continuent.tungsten.fsm.core.StateMachine.applyEvent(StateMachine.java:220)
INFO   | jvm 1    | 2014/06/13 18:14:16 |       at 
com.continuent.tungsten.fsm.event.EventProcessor.run(EventProcessor.java:78)
INFO   | jvm 1    | 2014/06/13 18:14:16 |       ... 5 more
INFO   | jvm 1    | 2014/06/13 18:14:16 | Caused by: 
com.continuent.tungsten.replicator.ReplicatorException: Unable to prepare 
plugin: class name=com.continuent.tungsten.replicator.thl.THL 
message=[java.sql.SQLException: ORA-06550: line , column :
INFO   | jvm 1    | 2014/06/13 18:14:16 | ORA-06512: at "SYS.DBMS_CDC_PUBLISH", 
line 999
INFO   | jvm 1    | 2014/06/13 18:14:16 | ORA-06512: at line 1
INFO   | jvm 1    | 2014/06/13 18:14:16 | ]
INFO   | jvm 1    | 2014/06/13 18:14:16 |       at 
com.continuent.tungsten.replicator.conf.ReplicatorRuntime.preparePlugin(Replicat
orRuntime.java:1259)
INFO   | jvm 1    | 2014/06/13 18:14:16 |       at 
com.continuent.tungsten.replicator.pipeline.Pipeline.prepare(Pipeline.java:266)
INFO   | jvm 1    | 2014/06/13 18:14:16 |       at 
com.continuent.tungsten.replicator.conf.ReplicatorRuntime.prepare(ReplicatorRunt
ime.java:642)
INFO   | jvm 1    | 2014/06/13 18:14:16 |       at 
com.continuent.tungsten.replicator.management.tungsten.TungstenPlugin.online(Tun
gstenPlugin.java:391)
INFO   | jvm 1    | 2014/06/13 18:14:16 |       at 
com.continuent.tungsten.replicator.management.OpenReplicatorManager$OfflineToSyn
chronizingAction.doAction(OpenReplicatorManager.java:1376)
INFO   | jvm 1    | 2014/06/13 18:14:16 |       ... 7 more
INFO   | jvm 1    | 2014/06/13 18:14:16 | Caused by: 
com.continuent.tungsten.replicator.thl.THLException: java.sql.SQLException: 
ORA-06550: line , column :
INFO   | jvm 1    | 2014/06/13 18:14:16 | ORA-06512: at "SYS.DBMS_CDC_PUBLISH", 
line 999
INFO   | jvm 1    | 2014/06/13 18:14:16 | ORA-06512: at line 1
INFO   | jvm 1    | 2014/06/13 18:14:16 | 
INFO   | jvm 1    | 2014/06/13 18:14:16 |       at 
com.continuent.tungsten.replicator.thl.CatalogManager.prepareSchema(CatalogManag
er.java:200)
INFO   | jvm 1    | 2014/06/13 18:14:16 |       at 
com.continuent.tungsten.replicator.thl.THL.prepare(THL.java:379)
INFO   | jvm 1    | 2014/06/13 18:14:16 |       at 
com.continuent.tungsten.replicator.conf.ReplicatorRuntime.preparePlugin(Replicat
orRuntime.java:1255)
INFO   | jvm 1    | 2014/06/13 18:14:16 |       ... 11 more
INFO   | jvm 1    | 2014/06/13 18:14:16 | Caused by: java.sql.SQLException: 
ORA-06550: line , column :
INFO   | jvm 1    | 2014/06/13 18:14:16 | ORA-06512: at "SYS.DBMS_CDC_PUBLISH", 
line 999
INFO   | jvm 1    | 2014/06/13 18:14:16 | ORA-06512: at line 1
INFO   | jvm 1    | 2014/06/13 18:14:16 | 
INFO   | jvm 1    | 2014/06/13 18:14:16 |       at 
oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:447)
INFO   | jvm 1    | 2014/06/13 18:14:16 |       at 
oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:396)
INFO   | jvm 1    | 2014/06/13 18:14:16 |       at 
oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:951)
INFO   | jvm 1    | 2014/06/13 18:14:16 |       at 
oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:513)
INFO   | jvm 1    | 2014/06/13 18:14:16 |       at 
oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:227)
INFO   | jvm 1    | 2014/06/13 18:14:16 |       at 
oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:531)
INFO   | jvm 1    | 2014/06/13 18:14:16 |       at 
oracle.jdbc.driver.T4CStatement.doOall8(T4CStatement.java:195)
INFO   | jvm 1    | 2014/06/13 18:14:16 |       at 
oracle.jdbc.driver.T4CStatement.executeForRows(T4CStatement.java:1036)
INFO   | jvm 1    | 2014/06/13 18:14:16 |       at 
oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:133
6)
INFO   | jvm 1    | 2014/06/13 18:14:16 |       at 
oracle.jdbc.driver.OracleStatement.executeInternal(OracleStatement.java:1916)
INFO   | jvm 1    | 2014/06/13 18:14:16 |       at 
oracle.jdbc.driver.OracleStatement.execute(OracleStatement.java:1878)
INFO   | jvm 1    | 2014/06/13 18:14:16 |       at 
oracle.jdbc.driver.OracleStatementWrapper.execute(OracleStatementWrapper.java:31
8)
INFO   | jvm 1    | 2014/06/13 18:14:16 |       at 
com.continuent.tungsten.replicator.database.AbstractDatabase.execute(AbstractDat
abase.java:455)
INFO   | jvm 1    | 2014/06/13 18:14:16 |       at 
com.continuent.tungsten.replicator.database.OracleDatabase.createChangeTable(Ora
cleDatabase.java:835)
INFO   | jvm 1    | 2014/06/13 18:14:16 |       at 
com.continuent.tungsten.replicator.database.OracleDatabase.createTable(OracleDat
abase.java:702)
INFO   | jvm 1    | 2014/06/13 18:14:16 |       at 
com.continuent.tungsten.replicator.heartbeat.HeartbeatTable.initializeHeartbeatT
able(HeartbeatTable.java:134)
INFO   | jvm 1    | 2014/06/13 18:14:16 |       at 
com.continuent.tungsten.replicator.thl.CatalogManager.prepareSchema(CatalogManag
er.java:186)
INFO   | jvm 1    | 2014/06/13 18:14:16 |       ... 13 more
INFO   | jvm 1    | 2014/06/13 18:14:16 | 2014-06-13 18:14:16,776 [shoe_master 
- pool-2-thread-1] WARN  management.OpenReplicatorManager Received irrelevant 
event for current state: state=OFFLINE:ERROR event=OfflineNotification

#####################

Grants given to the publisher / subscriber user:
GRANT "CONNECT" TO "FOOBAR_PUB";
GRANT "DBA" TO "FOOBAR_PUB";
GRANT "SELECT_CATALOG_ROLE" TO "FOOBAR_PUB";
GRANT "EXECUTE_CATALOG_ROLE" TO "FOOBAR_PUB";

GRANT ALL PRIVILEGES TO FOOBAR_PUB;

GRANT EXECUTE ON "SYS"."DBMS_CDC_PUBLISH" TO "FOOBAR_PUB";
GRANT EXECUTE ON "SYS"."DBMS_APPLY_ADM" TO "FOOBAR_PUB";
GRANT EXECUTE ON "SYS"."DBMS_CAPTURE_ADM" TO "FOOBAR_PUB";
GRANT EXECUTE ON "SYS"."DBMS_AQ_BQVIEW" TO "FOOBAR_PUB";

GRANT UNLIMITED TABLESPACE TO "FOOBAR_PUB";

exec DBMS_STREAMS_AUTH.GRANT_ADMIN_PRIVILEGE(GRANTEE => 'FOOBAR_PUB');
GRANT INSERT,UPDATE,DELETE ON SYS.CDC_CHANGE_TABLES$ TO FOOBAR_PUB;
GRANT SELECT ON SYS.CDC_CHANGE_TABLES$ TO FOOBAR_PUB WITH GRANT OPTION;

GRANT SELECT ON tungsten_load TO FOOBAR_PUB;
GRANT READ ON DIRECTORY tungsten_dir TO FOOBAR_PUB;

Extra grants given to the tungsten user:

grant execute on SYS.DBMS_CDC_PUBLISH to tungsten_foobar;
grant execute on SYS.DBMS_CAPTURE_ADM to tungsten_foobar;

Original issue reported on code.google.com by m...@chadwick.net on 13 Jun 2014 at 10:22

GoogleCodeExporter commented 9 years ago 
Quick question:  why are you running setupCDC on the slave--is this to enable 
multi-master replication?    Which tables are in the list to be published?  

Also, does the master come online successfully? 

Finally, which version of Oracle you using?

Original comment by robert.h...@continuent.com on 13 Jun 2014 at 10:30

GoogleCodeExporter commented 9 years ago 
> Quick question:  why are you running setupCDC on the slave--is this to enable 
multi-master replication?  

Because when I attempt to bring the slave online without having run setupCDC 
against it I get an error that ends with, "Did you run setupCDC.sh?"

> Which tables are in the list to be published?  

All tables within a schema.

> Also, does the master come online successfully? 

Based on a recent webinar we have kept the master in an offline state so we can 
use the parallel extractor to populate the slave. 

>Finally, which version of Oracle you using?

11g

Original comment by m...@chadwick.net on 13 Jun 2014 at 11:19