search

apache / seatunnel

SeaTunnel is a next-generation super high-performance, distributed, massive data integration tool.
https://seatunnel.apache.org/
Apache License 2.0
7.81k stars 1.76k forks source link

[Bug] [Oracle-CDC] CREATE TABLE LOG_MINING_FLUSH(LAST_SCN NUMBER(19,0)), Error Msg = ORA-01031: insufficient privileges #7352

Open luzongzhu opened 1 month ago

luzongzhu commented 1 month ago

Search before asking

What happened

CREATE TABLE LOG_MINING_FLUSH(LAST_SCN NUMBER(19,0)), OriginalSql = CREATE TABLE LOG_MINING_FLUSH(LAST_SCN NUMBER(19,0)), Error Msg = ORA-01031: insufficient privileges

SeaTunnel Version

2.3.6

SeaTunnel Config

env {
  job.name = "test1"
  job.mode = "STREAMING"
  checkpoint.interval = 10000
  checkpoint.timeout = 60000
  execution.parallelism = 1
}

source {
  Oracle-CDC {
    username = "logminer_user"
    password = "oracle"
    database-names = ["TEST101"]
    schema-names = ["AP"]
    table-names = ["TEST101.AP.AP_INVOICE_DISTRIBUTIONS_ALL","TEST101.AP.AP_SUPPLIERS","TEST101.AP.AP_INVOICES_ALL","TEST101.AP.AP_HOLDS_ALL","TEST101.AP.AP_PAYMENT_SCHEDULES_ALL","TEST101.AP.AP_SUPPLIER_SITES_ALL","TEST101.AP.AP_INVOICE_LINES_ALL","TEST101.AP.AP_AGING_PERIOD_LINES","TEST101.AP.AP_AGING_PERIODS"]
    base-url = "jdbc:oracle:thin:@xxxxx:1531:TEST101"
    source.reader.close.timeout = 120000
    connection.pool.size = 1
    startup.mode = "latest"
    debezium {
        database.oracle.jdbc.timezoneAsRegion = "false"
    }
    exactly_once = false
  }
}

sink {
  Doris {
    sink.enable-2pc = "false"
    fenodes = "xxxxx:8030"
    username = "root"
    password = "root123456"
    sink.enable-delete = "true"
    table.identifier = "ods.ODS_GTMS_${table_name}"
    sink.label-prefix = "sink"
    doris.config {
      format = "json"
      read_json_by_line = "true"
    }
  }
}

Running Command

/opt/soft/seatunnel/bin/seatunnel.sh --config /opt/soft/seatunnel/dwms/config/test1.conf -n test1

Error Exception

2024-08-09 11:52:04,288 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Loading configuration '/opt/soft/seatunnel/config/seatunnel.yaml' from System property 'seatunnel.config'
2024-08-09 11:52:04,293 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Using configuration file at /opt/soft/seatunnel/config/seatunnel.yaml
2024-08-09 11:52:04,296 INFO  [o.a.s.e.c.c.SeaTunnelConfig   ] [main] - seatunnel.home is /
2024-08-09 11:52:04,406 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Loading configuration '/opt/soft/seatunnel/config/hazelcast.yaml' from System property 'hazelcast.config'
2024-08-09 11:52:04,406 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Using configuration file at /opt/soft/seatunnel/config/hazelcast.yaml
2024-08-09 11:52:04,791 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Loading configuration '/opt/soft/seatunnel/config/hazelcast-client.yaml' from System property 'hazelcast.client.config'
2024-08-09 11:52:04,791 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Using configuration file at /opt/soft/seatunnel/config/hazelcast-client.yaml
2024-08-09 11:52:05,117 INFO  [.c.i.s.ClientInvocationService] [main] - hz.client_1 [seatunnel] [5.1] Running with 2 response threads, dynamic=true
2024-08-09 11:52:05,191 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is STARTING
2024-08-09 11:52:05,191 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is STARTED
2024-08-09 11:52:05,218 INFO  [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel] [5.1] Trying to connect to cluster: seatunnel
2024-08-09 11:52:05,226 INFO  [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel] [5.1] Trying to connect to [seatunnel]:5801
2024-08-09 11:52:05,261 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is CLIENT_CONNECTED
2024-08-09 11:52:05,262 INFO  [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel] [5.1] Authenticated with server [10.42.0.225]:5801:742431ee-0a3d-456e-ab2d-87deaacc21df, server version: 5.1, local address: /10.42.0.228:36932
2024-08-09 11:52:05,263 INFO  [c.h.i.d.Diagnostics           ] [main] - hz.client_1 [seatunnel] [5.1] Diagnostics disabled. To enable add -Dhazelcast.diagnostics.enabled=true to the JVM arguments.
2024-08-09 11:52:05,274 INFO  [c.h.c.i.s.ClientClusterService] [hz.client_1.event-5] - hz.client_1 [seatunnel] [5.1] 

Members [1] {
    Member [10.42.0.225]:5801 - 742431ee-0a3d-456e-ab2d-87deaacc21df
}

2024-08-09 11:52:05,310 INFO  [.c.i.s.ClientStatisticsService] [main] - Client statistics is enabled with period 5 seconds.
2024-08-09 11:52:05,490 INFO  [o.a.s.c.s.u.ConfigBuilder     ] [main] - Loading config file from path: /opt/soft/seatunnel/dwms/config/test1.conf
2024-08-09 11:52:05,550 INFO  [o.a.s.c.s.u.ConfigShadeUtils  ] [main] - Load config shade spi: [base64]
2024-08-09 11:52:05,632 INFO  [p.MultipleTableJobConfigParser] [main] - add common jar in plugins :[]
2024-08-09 11:52:05,650 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - Load SeaTunnelSink Plugin from /opt/soft/seatunnel/connectors
2024-08-09 11:52:05,654 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - Discovery plugin jar for: PluginIdentifier{engineType='seatunnel', pluginType='source', pluginName='Oracle-CDC'} at: file:/opt/soft/seatunnel/connectors/connector-cdc-oracle-2.3.6.jar
2024-08-09 11:52:05,654 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - Discovery plugin jar for: PluginIdentifier{engineType='seatunnel', pluginType='sink', pluginName='Doris'} at: file:/opt/soft/seatunnel/connectors/connector-doris-2.3.6.jar
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
2024-08-09 11:52:05,657 INFO  [p.MultipleTableJobConfigParser] [main] - start generating all sources.
2024-08-09 11:52:06,409 INFO  [.s.c.s.j.c.AbstractJdbcCatalog] [main] - Catalog Oracle established connection to jdbc:oracle:thin:@10.0.0.101:1531:TEST101
2024-08-09 11:57:38,043 INFO  [o.a.s.a.t.c.CatalogTableUtil  ] [main] - Get catalog tables, cost time: 332310 ms
2024-08-09 11:57:38,051 INFO  [.s.c.s.j.c.AbstractJdbcCatalog] [main] - Catalog Oracle closing
2024-08-09 11:57:38,314 INFO  [i.d.c.o.OracleConnection      ] [main] - Database Version: Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
2024-08-09 11:57:55,365 INFO  [.s.c.o.u.OracleConnectionUtils] [main] -      including 'TEST101.AP.AP_AGING_PERIOD_LINES' for further processing
2024-08-09 11:57:55,365 INFO  [.s.c.o.u.OracleConnectionUtils] [main] -      including 'TEST101.AP.AP_PAYMENT_SCHEDULES_ALL' for further processing
2024-08-09 11:57:55,367 INFO  [.s.c.o.u.OracleConnectionUtils] [main] -      including 'TEST101.AP.AP_INVOICE_DISTRIBUTIONS_ALL' for further processing
2024-08-09 11:57:55,369 INFO  [.s.c.o.u.OracleConnectionUtils] [main] -      including 'TEST101.AP.AP_INVOICE_LINES_ALL' for further processing
2024-08-09 11:57:55,397 INFO  [.s.c.o.u.OracleConnectionUtils] [main] -      including 'TEST101.AP.AP_AGING_PERIODS' for further processing
2024-08-09 11:57:55,401 INFO  [.s.c.o.u.OracleConnectionUtils] [main] -      including 'TEST101.AP.AP_SUPPLIERS' for further processing
2024-08-09 11:57:55,404 INFO  [.s.c.o.u.OracleConnectionUtils] [main] -      including 'TEST101.AP.AP_SUPPLIER_SITES_ALL' for further processing
2024-08-09 11:57:55,404 INFO  [.s.c.o.u.OracleConnectionUtils] [main] -      including 'TEST101.AP.AP_HOLDS_ALL' for further processing
2024-08-09 11:57:55,409 INFO  [.s.c.o.u.OracleConnectionUtils] [main] -      including 'TEST101.AP.AP_INVOICES_ALL' for further processing
2024-08-09 11:57:55,420 INFO  [i.d.j.JdbcConnection          ] [pool-2-thread-1] - Connection gracefully closed
2024-08-09 11:57:55,534 INFO  [i.d.c.o.OracleConnection      ] [main] - Database Version: Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
2024-08-09 11:58:03,704 INFO  [i.d.j.JdbcConnection          ] [pool-3-thread-1] - Connection gracefully closed
2024-08-09 11:58:03,718 INFO  [o.a.s.a.t.f.FactoryUtil       ] [main] - get the CatalogTable from source Oracle-CDC: Oracle.TEST101.AP.AP_INVOICE_DISTRIBUTIONS_ALL,Oracle.TEST101.AP.AP_SUPPLIERS,Oracle.TEST101.AP.AP_INVOICES_ALL,Oracle.TEST101.AP.AP_HOLDS_ALL,Oracle.TEST101.AP.AP_PAYMENT_SCHEDULES_ALL,Oracle.TEST101.AP.AP_SUPPLIER_SITES_ALL,Oracle.TEST101.AP.AP_INVOICE_LINES_ALL,Oracle.TEST101.AP.AP_AGING_PERIOD_LINES,Oracle.TEST101.AP.AP_AGING_PERIODS
2024-08-09 11:58:03,731 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - Load SeaTunnelSource Plugin from /opt/soft/seatunnel/connectors
2024-08-09 11:58:03,736 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - Discovery plugin jar for: PluginIdentifier{engineType='seatunnel', pluginType='source', pluginName='Oracle-CDC'} at: file:/opt/soft/seatunnel/connectors/connector-cdc-oracle-2.3.6.jar
2024-08-09 11:58:03,738 INFO  [p.MultipleTableJobConfigParser] [main] - start generating all transforms.
2024-08-09 11:58:03,738 INFO  [p.MultipleTableJobConfigParser] [main] - start generating all sinks.
2024-08-09 11:58:03,742 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - Load SeaTunnelSink Plugin from /opt/soft/seatunnel/connectors
2024-08-09 11:58:03,742 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - Discovery plugin jar for: PluginIdentifier{engineType='seatunnel', pluginType='sink', pluginName='Doris'} at: file:/opt/soft/seatunnel/connectors/connector-doris-2.3.6.jar
2024-08-09 11:58:03,750 INFO  [o.a.s.a.t.f.FactoryUtil       ] [main] - Create sink 'Doris' with upstream input catalog-table[database: TEST101, schema: AP, table: AP_INVOICE_DISTRIBUTIONS_ALL]
2024-08-09 11:58:03,757 INFO  [o.a.s.a.t.f.FactoryUtil       ] [main] - Create sink 'Doris' with upstream input catalog-table[database: TEST101, schema: AP, table: AP_SUPPLIERS]
2024-08-09 11:58:03,759 INFO  [o.a.s.a.t.f.FactoryUtil       ] [main] - Create sink 'Doris' with upstream input catalog-table[database: TEST101, schema: AP, table: AP_INVOICES_ALL]
2024-08-09 11:58:03,761 INFO  [o.a.s.a.t.f.FactoryUtil       ] [main] - Create sink 'Doris' with upstream input catalog-table[database: TEST101, schema: AP, table: AP_HOLDS_ALL]
2024-08-09 11:58:03,762 INFO  [o.a.s.a.t.f.FactoryUtil       ] [main] - Create sink 'Doris' with upstream input catalog-table[database: TEST101, schema: AP, table: AP_PAYMENT_SCHEDULES_ALL]
2024-08-09 11:58:03,766 INFO  [o.a.s.a.t.f.FactoryUtil       ] [main] - Create sink 'Doris' with upstream input catalog-table[database: TEST101, schema: AP, table: AP_SUPPLIER_SITES_ALL]
2024-08-09 11:58:03,768 INFO  [o.a.s.a.t.f.FactoryUtil       ] [main] - Create sink 'Doris' with upstream input catalog-table[database: TEST101, schema: AP, table: AP_INVOICE_LINES_ALL]
2024-08-09 11:58:03,772 INFO  [o.a.s.a.t.f.FactoryUtil       ] [main] - Create sink 'Doris' with upstream input catalog-table[database: TEST101, schema: AP, table: AP_AGING_PERIOD_LINES]
2024-08-09 11:58:03,773 INFO  [o.a.s.a.t.f.FactoryUtil       ] [main] - Create sink 'Doris' with upstream input catalog-table[database: TEST101, schema: AP, table: AP_AGING_PERIODS]
2024-08-09 11:58:03,873 INFO  [o.a.s.e.c.j.ClientJobProxy    ] [main] - Start submit job, job id: 874137939519471617, with plugin jar [file:/opt/soft/seatunnel/connectors/connector-cdc-oracle-2.3.6.jar, file:/opt/soft/seatunnel/connectors/connector-doris-2.3.6.jar]
2024-08-09 11:58:04,171 INFO  [o.a.s.e.c.j.ClientJobProxy    ] [main] - Submit job finished, job id: 874137939519471617, job name: test1
2024-08-09 11:58:04,188 WARN  [o.a.s.e.c.j.JobMetricsRunner  ] [job-metrics-runner-874137939519471617] - Failed to get job metrics summary, it maybe first-run
2024-08-09 11:59:04,191 INFO  [o.a.s.e.c.j.JobMetricsRunner  ] [job-metrics-runner-874137939519471617] - 
***********************************************
           Job Progress Information
***********************************************
Job Id                    :  874137939519471617
Read Count So Far         :                   0
Write Count So Far        :                   0
Average Read Count        :                 0/s
Average Write Count       :                 0/s
Last Statistic Time       : 2024-08-09 11:58:04
Current Statistic Time    : 2024-08-09 11:59:04
***********************************************

2024-08-09 12:00:01,610 INFO  [o.a.s.e.c.j.ClientJobProxy    ] [main] - Job (874137939519471617) end with state FAILED
2024-08-09 12:00:01,611 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is SHUTTING_DOWN
2024-08-09 12:00:01,617 INFO  [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel] [5.1] Removed connection to endpoint: [10.42.0.225]:5801:742431ee-0a3d-456e-ab2d-87deaacc21df, connection: ClientConnection{alive=false, connectionId=1, channel=NioChannel{/10.42.0.228:36932->seatunnel/10.43.80.120:5801}, remoteAddress=[10.42.0.225]:5801, lastReadTime=2024-08-09 12:00:01.599, lastWriteTime=2024-08-09 12:00:00.314, closedTime=2024-08-09 12:00:01.614, connected server version=5.1}
2024-08-09 12:00:01,617 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is CLIENT_DISCONNECTED
2024-08-09 12:00:01,622 ERROR [o.a.s.c.s.SeaTunnel           ] [main] - 

===============================================================================

2024-08-09 12:00:01,622 ERROR [o.a.s.c.s.SeaTunnel           ] [main] - Fatal Error, 

2024-08-09 12:00:01,622 ERROR [o.a.s.c.s.SeaTunnel           ] [main] - Please submit bug report in https://github.com/apache/seatunnel/issues
2024-08-09 12:00:01,620 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is SHUTDOWN
2024-08-09 12:00:01,622 INFO  [s.c.s.s.c.ClientExecuteCommand] [main] - Closed SeaTunnel client......
2024-08-09 12:00:01,622 INFO  [s.c.s.s.c.ClientExecuteCommand] [main] - Closed metrics executor service ......

2024-08-09 12:00:01,622 ERROR [o.a.s.c.s.SeaTunnel           ] [main] - Reason:SeaTunnel job executed failed 

2024-08-09 12:00:01,623 ERROR [o.a.s.c.s.SeaTunnel           ] [main] - Exception StackTrace:org.apache.seatunnel.core.starter.exception.CommandExecuteException: SeaTunnel job executed failed
    at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:211)
    at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40)
    at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:34)
Caused by: org.apache.seatunnel.engine.common.exception.SeaTunnelEngineException: java.lang.RuntimeException: One or more fetchers have encountered exception
    at org.apache.seatunnel.connectors.seatunnel.common.source.reader.fetcher.SplitFetcherManager.checkErrors(SplitFetcherManager.java:147)
    at org.apache.seatunnel.connectors.seatunnel.common.source.reader.SourceReaderBase.getNextFetch(SourceReaderBase.java:167)
    at org.apache.seatunnel.connectors.seatunnel.common.source.reader.SourceReaderBase.pollNext(SourceReaderBase.java:93)
    at org.apache.seatunnel.connectors.cdc.base.source.reader.IncrementalSourceReader.pollNext(IncrementalSourceReader.java:119)
    at org.apache.seatunnel.engine.server.task.flow.SourceFlowLifeCycle.collect(SourceFlowLifeCycle.java:156)
    at org.apache.seatunnel.engine.server.task.SourceSeaTunnelTask.collect(SourceSeaTunnelTask.java:127)
    at org.apache.seatunnel.engine.server.task.SeaTunnelTask.stateProcess(SeaTunnelTask.java:168)
    at org.apache.seatunnel.engine.server.task.SourceSeaTunnelTask.call(SourceSeaTunnelTask.java:132)
    at org.apache.seatunnel.engine.server.TaskExecutionService$BlockingWorker.run(TaskExecutionService.java:703)
    at org.apache.seatunnel.engine.server.TaskExecutionService$NamedTaskWrapper.run(TaskExecutionService.java:1021)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:750)
Caused by: java.lang.RuntimeException: SplitFetcher thread 0 received unexpected exception while polling the records
    at org.apache.seatunnel.connectors.seatunnel.common.source.reader.fetcher.SplitFetcher.runOnce(SplitFetcher.java:165)
    at org.apache.seatunnel.connectors.seatunnel.common.source.reader.fetcher.SplitFetcher.run(SplitFetcher.java:81)
    ... 5 more
Caused by: org.apache.kafka.connect.errors.ConnectException: An exception occurred in the change event producer. This connector will be stopped.
    at io.debezium.pipeline.ErrorHandler.setProducerThrowable(ErrorHandler.java:50)
    at io.debezium.connector.oracle.logminer.LogMinerStreamingChangeEventSource.execute(LogMinerStreamingChangeEventSource.java:264)
    at org.apache.seatunnel.connectors.seatunnel.cdc.oracle.source.reader.fetch.logminer.OracleRedoLogFetchTask$RedoLogSplitReadTask.execute(OracleRedoLogFetchTask.java:147)
    at org.apache.seatunnel.connectors.seatunnel.cdc.oracle.source.reader.fetch.logminer.OracleRedoLogFetchTask.execute(OracleRedoLogFetchTask.java:73)
    at org.apache.seatunnel.connectors.cdc.base.source.reader.external.IncrementalSourceStreamFetcher.lambda$submitTask$0(IncrementalSourceStreamFetcher.java:106)
    ... 5 more
Caused by: io.debezium.DebeziumException: Failed to create flush table
    at io.debezium.connector.oracle.logminer.logwriter.CommitLogWriterFlushStrategy.createFlushTableIfNotExists(CommitLogWriterFlushStrategy.java:111)
    at io.debezium.connector.oracle.logminer.logwriter.CommitLogWriterFlushStrategy.<init>(CommitLogWriterFlushStrategy.java:47)
    at io.debezium.connector.oracle.logminer.LogMinerStreamingChangeEventSource.resolveFlushStrategy(LogMinerStreamingChangeEventSource.java:1020)
    at io.debezium.connector.oracle.logminer.LogMinerStreamingChangeEventSource.execute(LogMinerStreamingChangeEventSource.java:165)
    ... 8 more
Caused by: java.sql.SQLSyntaxErrorException: ORA-01031: insufficient privileges

    at oracle.jdbc.driver.T4CTTIoer11.processError(T4CTTIoer11.java:494)
    at oracle.jdbc.driver.T4CTTIoer11.processError(T4CTTIoer11.java:446)
    at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:1054)
    at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:623)
    at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:252)
    at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:612)
    at oracle.jdbc.driver.T4CStatement.doOall8(T4CStatement.java:213)
    at oracle.jdbc.driver.T4CStatement.doOall8(T4CStatement.java:37)
    at oracle.jdbc.driver.T4CStatement.executeForRows(T4CStatement.java:896)
    at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1119)
    at oracle.jdbc.driver.OracleStatement.executeInternal(OracleStatement.java:1737)
    at oracle.jdbc.driver.OracleStatement.execute(OracleStatement.java:1692)
    at oracle.jdbc.driver.OracleStatementWrapper.execute(OracleStatementWrapper.java:300)
    at io.debezium.jdbc.JdbcConnection.executeWithoutCommitting(JdbcConnection.java:1446)
    at io.debezium.connector.oracle.logminer.logwriter.CommitLogWriterFlushStrategy.createFlushTableIfNotExists(CommitLogWriterFlushStrategy.java:100)
    ... 11 more
Caused by: Error : 1031, Position : 0, Sql = CREATE TABLE LOG_MINING_FLUSH(LAST_SCN NUMBER(19,0)), OriginalSql = CREATE TABLE LOG_MINING_FLUSH(LAST_SCN NUMBER(19,0)), Error Msg = ORA-01031: insufficient privileges

    at oracle.jdbc.driver.T4CTTIoer11.processError(T4CTTIoer11.java:498)
    ... 25 more

    at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:203)
    ... 2 more

2024-08-09 12:00:01,629 ERROR [o.a.s.c.s.SeaTunnel           ] [main] - 
===============================================================================

Exception in thread "main" org.apache.seatunnel.core.starter.exception.CommandExecuteException: SeaTunnel job executed failed
    at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:211)
    at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40)
    at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:34)
Caused by: org.apache.seatunnel.engine.common.exception.SeaTunnelEngineException: java.lang.RuntimeException: One or more fetchers have encountered exception
    at org.apache.seatunnel.connectors.seatunnel.common.source.reader.fetcher.SplitFetcherManager.checkErrors(SplitFetcherManager.java:147)
    at org.apache.seatunnel.connectors.seatunnel.common.source.reader.SourceReaderBase.getNextFetch(SourceReaderBase.java:167)
    at org.apache.seatunnel.connectors.seatunnel.common.source.reader.SourceReaderBase.pollNext(SourceReaderBase.java:93)
    at org.apache.seatunnel.connectors.cdc.base.source.reader.IncrementalSourceReader.pollNext(IncrementalSourceReader.java:119)
    at org.apache.seatunnel.engine.server.task.flow.SourceFlowLifeCycle.collect(SourceFlowLifeCycle.java:156)
    at org.apache.seatunnel.engine.server.task.SourceSeaTunnelTask.collect(SourceSeaTunnelTask.java:127)
    at org.apache.seatunnel.engine.server.task.SeaTunnelTask.stateProcess(SeaTunnelTask.java:168)
    at org.apache.seatunnel.engine.server.task.SourceSeaTunnelTask.call(SourceSeaTunnelTask.java:132)
    at org.apache.seatunnel.engine.server.TaskExecutionService$BlockingWorker.run(TaskExecutionService.java:703)
    at org.apache.seatunnel.engine.server.TaskExecutionService$NamedTaskWrapper.run(TaskExecutionService.java:1021)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:750)
Caused by: java.lang.RuntimeException: SplitFetcher thread 0 received unexpected exception while polling the records
    at org.apache.seatunnel.connectors.seatunnel.common.source.reader.fetcher.SplitFetcher.runOnce(SplitFetcher.java:165)
    at org.apache.seatunnel.connectors.seatunnel.common.source.reader.fetcher.SplitFetcher.run(SplitFetcher.java:81)
    ... 5 more
Caused by: org.apache.kafka.connect.errors.ConnectException: An exception occurred in the change event producer. This connector will be stopped.
    at io.debezium.pipeline.ErrorHandler.setProducerThrowable(ErrorHandler.java:50)
    at io.debezium.connector.oracle.logminer.LogMinerStreamingChangeEventSource.execute(LogMinerStreamingChangeEventSource.java:264)
    at org.apache.seatunnel.connectors.seatunnel.cdc.oracle.source.reader.fetch.logminer.OracleRedoLogFetchTask$RedoLogSplitReadTask.execute(OracleRedoLogFetchTask.java:147)
    at org.apache.seatunnel.connectors.seatunnel.cdc.oracle.source.reader.fetch.logminer.OracleRedoLogFetchTask.execute(OracleRedoLogFetchTask.java:73)
    at org.apache.seatunnel.connectors.cdc.base.source.reader.external.IncrementalSourceStreamFetcher.lambda$submitTask$0(IncrementalSourceStreamFetcher.java:106)
    ... 5 more
Caused by: io.debezium.DebeziumException: Failed to create flush table
    at io.debezium.connector.oracle.logminer.logwriter.CommitLogWriterFlushStrategy.createFlushTableIfNotExists(CommitLogWriterFlushStrategy.java:111)
    at io.debezium.connector.oracle.logminer.logwriter.CommitLogWriterFlushStrategy.<init>(CommitLogWriterFlushStrategy.java:47)
    at io.debezium.connector.oracle.logminer.LogMinerStreamingChangeEventSource.resolveFlushStrategy(LogMinerStreamingChangeEventSource.java:1020)
    at io.debezium.connector.oracle.logminer.LogMinerStreamingChangeEventSource.execute(LogMinerStreamingChangeEventSource.java:165)
    ... 8 more
Caused by: java.sql.SQLSyntaxErrorException: ORA-01031: insufficient privileges

    at oracle.jdbc.driver.T4CTTIoer11.processError(T4CTTIoer11.java:494)
    at oracle.jdbc.driver.T4CTTIoer11.processError(T4CTTIoer11.java:446)
    at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:1054)
    at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:623)
    at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:252)
    at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:612)
    at oracle.jdbc.driver.T4CStatement.doOall8(T4CStatement.java:213)
    at oracle.jdbc.driver.T4CStatement.doOall8(T4CStatement.java:37)
2024-08-09 12:00:01,779 INFO  [s.c.s.s.c.ClientExecuteCommand] [Thread-10] - run shutdown hook because get close signal
    at oracle.jdbc.driver.T4CStatement.executeForRows(T4CStatement.java:896)
    at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1119)
    at oracle.jdbc.driver.OracleStatement.executeInternal(OracleStatement.java:1737)
    at oracle.jdbc.driver.OracleStatement.execute(OracleStatement.java:1692)
    at oracle.jdbc.driver.OracleStatementWrapper.execute(OracleStatementWrapper.java:300)
    at io.debezium.jdbc.JdbcConnection.executeWithoutCommitting(JdbcConnection.java:1446)
    at io.debezium.connector.oracle.logminer.logwriter.CommitLogWriterFlushStrategy.createFlushTableIfNotExists(CommitLogWriterFlushStrategy.java:100)
    ... 11 more
Caused by: Error : 1031, Position : 0, Sql = CREATE TABLE LOG_MINING_FLUSH(LAST_SCN NUMBER(19,0)), OriginalSql = CREATE TABLE LOG_MINING_FLUSH(LAST_SCN NUMBER(19,0)), Error Msg = ORA-01031: insufficient privileges

    at oracle.jdbc.driver.T4CTTIoer11.processError(T4CTTIoer11.java:498)
    ... 25 more

    at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:203)
    ... 2 more

Zeta or Flink or Spark Version

Zeta

Java or Scala Version

1.8

Screenshots

No response

Are you willing to submit PR?

Code of Conduct

hailin0 commented 1 month ago

cc @dailai

luzongzhu commented 1 month ago

Logminer_user configured according to SeaTunnel Oracle CDC documentation

dailai commented 1 month ago

Insufficient permissions, please try authorization： grant resource to user

luzongzhu commented 1 month ago

Insufficient permissions, please try authorization： grant resource to user @dailai logminer_user configured according to SeaTunnel Oracle CDC documentation，The document does not mention the need to grant resources to users

luzongzhu commented 1 month ago

Oracle-CDC version 2.3.6 has updated the dbz version, and you need to add the following permissions to logminer_user before it can be used.

GRANT RESOURCE TO logminer_user;
GRANT SELECT ON V_$MYSTAT TO logminer_user;
GRANT SELECT ON V_$STATNAME TO logminer_user;
dailai commented 1 month ago

Oracle-CDC version 2.3.6 has updated the dbz version, and you need to add the following permissions to logminer_user before it can be used.

GRANT RESOURCE TO logminer_user;
GRANT SELECT ON V_$MYSTAT TO logminer_user;
GRANT SELECT ON V_$STATNAME TO logminer_user;

Thank you for your suggestion. We are discussing whether we can remove the creation of temporary tables in logminer.

github-actions[bot] commented 1 week ago

This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.