search

apache / seatunnel

SeaTunnel is a next-generation super high-performance, distributed, massive data integration tool.
https://seatunnel.apache.org/
Apache License 2.0
7.82k stars 1.76k forks source link

[Bug] Can not find catalog table with factoryId [Oracle] #7304

Open cavellpotter opened 1 month ago

cavellpotter commented 1 month ago

Search before asking

What happened

同步oracle数据到starrocks报错

SeaTunnel Version

2.3.5

SeaTunnel Config

env {
  parallelism = 2
  job.mode = "STREAMING"
  checkpoint.interval = 2000
}
source {
  Oracle-CDC {
    result_table_name = "TEST_STARROCKS"
    username = "c##dbzuser"
    password = "dbz"
    database-names = ["ORACLEDB"]
    schema-names = ["c##dbzuser"]
    table-names = ["ORACLEDB.c##dbzuser.TEST_STARROCKS"]
    base-url = "jdbc:oracle:thin:@192.168.31.187:1521:ORACLEDB"
    source.reader.close.timeout = 1200000
  }
}
sink {
  StarRocks {
    nodeUrls = ["192.168.31.187:8030"]
    username = root
    password = ""
    database = "test"
    table = "test_seatunnel"
    enable_upsert_delete = true
    source_table_name = ["TEST_STARROCKS"]
  }
}

Running Command

./bin/seatunnel.sh --config ./config/v2.streaming.conf.template -e local

Error Exception

******************************************************************************************
-e and --deploy-mode deprecated in 2.3.1, please use -m and --master instead of it
******************************************************************************************
八月 04, 2024 1:32:12 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Loading configuration '/home/apache-seatunnel-2.3.5/config/seatunnel.yaml' from System property 'seatunnel.config'
八月 04, 2024 1:32:12 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Using configuration file at /home/apache-seatunnel-2.3.5/config/seatunnel.yaml
八月 04, 2024 1:32:12 上午 org.apache.seatunnel.engine.common.config.SeaTunnelConfig
信息: seatunnel.home is /home/apache-seatunnel-2.3.5
八月 04, 2024 1:32:12 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Loading configuration '/home/apache-seatunnel-2.3.5/config/hazelcast.yaml' from System property 'hazelcast.config'
八月 04, 2024 1:32:12 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Using configuration file at /home/apache-seatunnel-2.3.5/config/hazelcast.yaml
八月 04, 2024 1:32:12 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Loading configuration '/home/apache-seatunnel-2.3.5/config/hazelcast-client.yaml' from System property 'hazelcast.client.config'
八月 04, 2024 1:32:12 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Using configuration file at /home/apache-seatunnel-2.3.5/config/hazelcast-client.yaml
2024-08-04 01:32:12,657 WARN  [c.h.i.AddressPicker           ] [main] - [LOCAL] [seatunnel-651771] [5.1] You configured your member address as host name. Please be aware of that your dns can be spoofed. Make sure that your dns configurations are correct.
2024-08-04 01:32:12,657 INFO  [c.h.i.AddressPicker           ] [main] - [LOCAL] [seatunnel-651771] [5.1] Resolving domain name 'localhost' to address(es): [127.0.0.1, 0:0:0:0:0:0:0:1]
2024-08-04 01:32:12,658 INFO  [c.h.i.AddressPicker           ] [main] - [LOCAL] [seatunnel-651771] [5.1] Interfaces is disabled, trying to pick one address from TCP-IP config addresses: [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1]
2024-08-04 01:32:12,672 INFO  [o.a.s.e.s.SeaTunnelServer     ] [main] - SeaTunnel server start...
2024-08-04 01:32:12,673 INFO  [c.h.system                    ] [main] - [localhost]:5801 [seatunnel-651771] [5.1] Based on Hazelcast IMDG version: 5.1.0 (20220228 - 21f20e7)
2024-08-04 01:32:12,673 INFO  [c.h.system                    ] [main] - [localhost]:5801 [seatunnel-651771] [5.1] Cluster name: seatunnel-651771
2024-08-04 01:32:12,673 INFO  [c.h.system                    ] [main] - [localhost]:5801 [seatunnel-651771] [5.1] 

 _____               _____                             _ 
/  ___|             |_   _|                           | |
\ `--.   ___   __ _   | |   _   _  _ __   _ __    ___ | |
 `--. \ / _ \ / _` |  | |  | | | || '_ \ | '_ \  / _ \| |
/\__/ /|  __/| (_| |  | |  | |_| || | | || | | ||  __/| |
\____/  \___| \__,_|  \_/   \__,_||_| |_||_| |_| \___||_|

2024-08-04 01:32:12,673 INFO  [c.h.system                    ] [main] - [localhost]:5801 [seatunnel-651771] [5.1] Copyright © 2021-2022 The Apache Software Foundation. Apache SeaTunnel, SeaTunnel, and its feather logo are trademarks of The Apache Software Foundation.
2024-08-04 01:32:12,673 INFO  [c.h.system                    ] [main] - [localhost]:5801 [seatunnel-651771] [5.1] Integrity Checker is disabled. Fail-fast on corrupted executables will not be performed.
To enable integrity checker do one of the following: 
  - Change member config using Java API: config.setIntegrityCheckerEnabled(true);
  - Change XML/YAML configuration property: Set hazelcast.integrity-checker.enabled to true
  - Add system property: -Dhz.integritychecker.enabled=true (for Hazelcast embedded, works only when loading config via Config.load)
  - Add environment variable: HZ_INTEGRITYCHECKER_ENABLED=true (recommended when running container image. For Hazelcast embedded, works only when loading config via Config.load)
2024-08-04 01:32:12,675 INFO  [c.h.system                    ] [main] - [localhost]:5801 [seatunnel-651771] [5.1] The Jet engine is disabled.
To enable the Jet engine on the members, do one of the following:
  - Change member config using Java API: config.getJetConfig().setEnabled(true)
  - Change XML/YAML configuration property: Set hazelcast.jet.enabled to true
  - Add system property: -Dhz.jet.enabled=true (for Hazelcast embedded, works only when loading config via Config.load)
  - Add environment variable: HZ_JET_ENABLED=true (recommended when running container image. For Hazelcast embedded, works only when loading config via Config.load)
2024-08-04 01:32:12,911 INFO  [c.h.s.security                ] [main] - [localhost]:5801 [seatunnel-651771] [5.1] Enable DEBUG/FINE log level for log category com.hazelcast.system.security  or use -Dhazelcast.security.recommendations system property to see 🔒 security recommendations and the status of current config.
2024-08-04 01:32:12,950 INFO  [c.h.i.i.Node                  ] [main] - [localhost]:5801 [seatunnel-651771] [5.1] Using TCP/IP discovery
2024-08-04 01:32:12,950 WARN  [c.h.c.CPSubsystem             ] [main] - [localhost]:5801 [seatunnel-651771] [5.1] CP Subsystem is not enabled. CP data structures will operate in UNSAFE mode! Please note that UNSAFE mode will not provide strong consistency guarantees.
2024-08-04 01:32:13,014 INFO  [.c.c.DefaultClassLoaderService] [main] - start classloader service
八月 04, 2024 1:32:13 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Loading configuration '/home/apache-seatunnel-2.3.5/config/seatunnel.yaml' from System property 'seatunnel.config'
八月 04, 2024 1:32:13 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Using configuration file at /home/apache-seatunnel-2.3.5/config/seatunnel.yaml
八月 04, 2024 1:32:13 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Loading configuration '/home/apache-seatunnel-2.3.5/config/hazelcast.yaml' from System property 'hazelcast.config'
八月 04, 2024 1:32:13 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Using configuration file at /home/apache-seatunnel-2.3.5/config/hazelcast.yaml
2024-08-04 01:32:13,024 WARN  [o.a.s.e.s.TaskExecutionService] [pool-2-thread-1] - [localhost]:5801 [seatunnel-651771] [5.1] The Node is not ready yet, Node state STARTING,looking forward to the next scheduling
2024-08-04 01:32:13,025 INFO  [o.a.s.e.s.TaskExecutionService] [main] - [localhost]:5801 [seatunnel-651771] [5.1] Created new BusWork : 25916650
2024-08-04 01:32:13,033 WARN  [a.s.e.s.s.s.DefaultSlotService] [hz.main.seaTunnel.slotService.thread] - failed send heartbeat to resource manager, will retry later. this address: [localhost]:5801
2024-08-04 01:32:13,035 INFO  [o.a.s.e.s.CoordinatorService  ] [pool-6-thread-1] - [localhost]:5801 [seatunnel-651771] [5.1] 
***********************************************
     CoordinatorService Thread Pool Status
***********************************************
activeCount               :                   0
corePoolSize              :                   0
maximumPoolSize           :          2147483647
poolSize                  :                   0
completedTaskCount        :                   0
taskCount                 :                   0
***********************************************

2024-08-04 01:32:13,055 INFO  [c.h.i.d.Diagnostics           ] [main] - [localhost]:5801 [seatunnel-651771] [5.1] Diagnostics disabled. To enable add -Dhazelcast.diagnostics.enabled=true to the JVM arguments.
2024-08-04 01:32:13,058 INFO  [c.h.c.LifecycleService        ] [main] - [localhost]:5801 [seatunnel-651771] [5.1] [localhost]:5801 is STARTING
2024-08-04 01:32:13,077 INFO  [c.h.i.c.i.TcpIpJoiner         ] [hz.main.cached.thread-15] - [localhost]:5801 [seatunnel-651771] [5.1] [localhost]:5822 is added to the blacklist.
2024-08-04 01:32:13,077 INFO  [c.h.i.c.i.TcpIpJoiner         ] [hz.main.cached.thread-13] - [localhost]:5801 [seatunnel-651771] [5.1] [localhost]:5829 is added to the blacklist.
2024-08-04 01:32:13,077 INFO  [c.h.i.c.i.TcpIpJoiner         ] [hz.main.cached.thread-10] - [localhost]:5801 [seatunnel-651771] [5.1] [localhost]:5804 is added to the blacklist.
2024-08-04 01:32:13,077 INFO  [c.h.i.c.i.TcpIpJoiner         ] [hz.main.cached.thread-11] - [localhost]:5801 [seatunnel-651771] [5.1] [localhost]:5810 is added to the blacklist.
2024-08-04 01:32:13,077 INFO  [c.h.i.c.i.TcpIpJoiner         ] [hz.main.cached.thread-8] - [localhost]:5801 [seatunnel-651771] [5.1] [localhost]:5817 is added to the blacklist.
2024-08-04 01:32:13,077 INFO  [c.h.i.c.i.TcpIpJoiner         ] [hz.main.cached.thread-2] - [localhost]:5801 [seatunnel-651771] [5.1] [localhost]:5821 is added to the blacklist.
2024-08-04 01:32:13,077 INFO  [c.h.i.c.i.TcpIpJoiner         ] [hz.main.cached.thread-5] - [localhost]:5801 [seatunnel-651771] [5.1] [localhost]:5815 is added to the blacklist.
2024-08-04 01:32:13,077 INFO  [c.h.i.c.i.TcpIpJoiner         ] [hz.main.cached.thread-1] - [localhost]:5801 [seatunnel-651771] [5.1] [localhost]:5827 is added to the blacklist.
2024-08-04 01:32:13,077 INFO  [c.h.i.c.i.TcpIpJoiner         ] [hz.main.cached.thread-15] - [localhost]:5801 [seatunnel-651771] [5.1] [localhost]:5826 is added to the blacklist.
2024-08-04 01:32:13,077 INFO  [c.h.i.c.i.TcpIpJoiner         ] [hz.main.cached.thread-7] - [localhost]:5801 [seatunnel-651771] [5.1] [localhost]:5819 is added to the blacklist.
2024-08-04 01:32:13,077 INFO  [c.h.i.c.i.TcpIpJoiner         ] [hz.main.cached.thread-14] - [localhost]:5801 [seatunnel-651771] [5.1] [localhost]:5802 is added to the blacklist.
2024-08-04 01:32:13,077 INFO  [c.h.i.c.i.TcpIpJoiner         ] [hz.main.cached.thread-10] - [localhost]:5801 [seatunnel-651771] [5.1] [localhost]:5824 is added to the blacklist.
2024-08-04 01:32:13,078 INFO  [c.h.i.c.i.TcpIpJoiner         ] [hz.main.cached.thread-8] - [localhost]:5801 [seatunnel-651771] [5.1] [localhost]:5814 is added to the blacklist.
2024-08-04 01:32:13,077 INFO  [c.h.i.c.i.TcpIpJoiner         ] [hz.main.cached.thread-3] - [localhost]:5801 [seatunnel-651771] [5.1] [localhost]:5823 is added to the blacklist.
2024-08-04 01:32:13,077 INFO  [c.h.i.c.i.TcpIpJoiner         ] [hz.main.cached.thread-16] - [localhost]:5801 [seatunnel-651771] [5.1] [localhost]:5820 is added to the blacklist.
2024-08-04 01:32:13,078 INFO  [c.h.i.c.i.TcpIpJoiner         ] [hz.main.cached.thread-1] - [localhost]:5801 [seatunnel-651771] [5.1] [localhost]:5816 is added to the blacklist.
2024-08-04 01:32:13,078 INFO  [c.h.i.c.i.TcpIpJoiner         ] [hz.main.cached.thread-10] - [localhost]:5801 [seatunnel-651771] [5.1] [localhost]:5805 is added to the blacklist.
2024-08-04 01:32:13,077 INFO  [c.h.i.c.i.TcpIpJoiner         ] [hz.main.cached.thread-9] - [localhost]:5801 [seatunnel-651771] [5.1] [localhost]:5806 is added to the blacklist.
2024-08-04 01:32:13,078 INFO  [c.h.i.c.i.TcpIpJoiner         ] [hz.main.cached.thread-11] - [localhost]:5801 [seatunnel-651771] [5.1] [localhost]:5812 is added to the blacklist.
2024-08-04 01:32:13,078 INFO  [c.h.i.c.i.TcpIpJoiner         ] [hz.main.cached.thread-2] - [localhost]:5801 [seatunnel-651771] [5.1] [localhost]:5818 is added to the blacklist.
2024-08-04 01:32:13,077 INFO  [c.h.i.c.i.TcpIpJoiner         ] [hz.main.cached.thread-12] - [localhost]:5801 [seatunnel-651771] [5.1] [localhost]:5808 is added to the blacklist.
2024-08-04 01:32:13,078 INFO  [c.h.i.c.i.TcpIpJoiner         ] [hz.main.cached.thread-16] - [localhost]:5801 [seatunnel-651771] [5.1] [localhost]:5828 is added to the blacklist.
2024-08-04 01:32:13,077 INFO  [c.h.i.c.i.TcpIpJoiner         ] [hz.main.cached.thread-6] - [localhost]:5801 [seatunnel-651771] [5.1] [localhost]:5813 is added to the blacklist.
2024-08-04 01:32:13,078 INFO  [c.h.i.c.i.TcpIpJoiner         ] [hz.main.cached.thread-14] - [localhost]:5801 [seatunnel-651771] [5.1] [localhost]:5811 is added to the blacklist.
2024-08-04 01:32:13,078 INFO  [c.h.i.c.i.TcpIpJoiner         ] [hz.main.cached.thread-8] - [localhost]:5801 [seatunnel-651771] [5.1] [localhost]:5809 is added to the blacklist.
2024-08-04 01:32:13,078 INFO  [c.h.i.c.i.TcpIpJoiner         ] [hz.main.cached.thread-15] - [localhost]:5801 [seatunnel-651771] [5.1] [localhost]:5807 is added to the blacklist.
2024-08-04 01:32:13,077 INFO  [c.h.i.c.i.TcpIpJoiner         ] [hz.main.cached.thread-4] - [localhost]:5801 [seatunnel-651771] [5.1] [localhost]:5825 is added to the blacklist.
2024-08-04 01:32:13,078 INFO  [c.h.i.c.i.TcpIpJoiner         ] [hz.main.cached.thread-3] - [localhost]:5801 [seatunnel-651771] [5.1] [localhost]:5830 is added to the blacklist.
2024-08-04 01:32:13,078 INFO  [c.h.i.c.i.TcpIpJoiner         ] [hz.main.cached.thread-10] - [localhost]:5801 [seatunnel-651771] [5.1] [localhost]:5803 is added to the blacklist.
2024-08-04 01:32:14,078 INFO  [c.h.i.c.ClusterService        ] [main] - [localhost]:5801 [seatunnel-651771] [5.1] 

Members {size:1, ver:1} [
        Member [localhost]:5801 - b2440996-aec2-4c80-8d60-33d8f84dbeab this
]

2024-08-04 01:32:14,082 INFO  [c.h.c.LifecycleService        ] [main] - [localhost]:5801 [seatunnel-651771] [5.1] [localhost]:5801 is STARTED
2024-08-04 01:32:14,121 INFO  [.c.i.s.ClientInvocationService] [main] - hz.client_1 [seatunnel-651771] [5.1] Running with 2 response threads, dynamic=true
2024-08-04 01:32:14,127 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel-651771] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is STARTING
2024-08-04 01:32:14,128 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel-651771] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is STARTED
2024-08-04 01:32:14,132 INFO  [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel-651771] [5.1] Trying to connect to cluster: seatunnel-651771
2024-08-04 01:32:14,133 INFO  [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel-651771] [5.1] Trying to connect to [localhost]:5801
2024-08-04 01:32:14,134 INFO  [o.a.s.e.s.CoordinatorService  ] [pool-5-thread-1] - [localhost]:5801 [seatunnel-651771] [5.1] This node become a new active master node, begin init coordinator service
2024-08-04 01:32:14,140 INFO  [.p.t.AuthenticationMessageTask] [hz.main.priority-generic-operation.thread-0] - [localhost]:5801 [seatunnel-651771] [5.1] Received auth from Connection[id=1, /127.0.0.1:5801->/127.0.0.1:52193, qualifier=null, endpoint=[127.0.0.1]:52193, remoteUuid=29d4a84b-c759-4cb5-beca-db775ef7af09, alive=true, connectionType=JVM, planeIndex=-1], successfully authenticated, clientUuid: 29d4a84b-c759-4cb5-beca-db775ef7af09, client name: hz.client_1, client version: 5.1
2024-08-04 01:32:14,141 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel-651771] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is CLIENT_CONNECTED
2024-08-04 01:32:14,141 INFO  [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel-651771] [5.1] Authenticated with server [localhost]:5801:b2440996-aec2-4c80-8d60-33d8f84dbeab, server version: 5.1, local address: /127.0.0.1:52193
2024-08-04 01:32:14,142 INFO  [c.h.i.d.Diagnostics           ] [main] - hz.client_1 [seatunnel-651771] [5.1] Diagnostics disabled. To enable add -Dhazelcast.diagnostics.enabled=true to the JVM arguments.
2024-08-04 01:32:14,146 INFO  [c.h.c.i.s.ClientClusterService] [hz.client_1.event-9] - hz.client_1 [seatunnel-651771] [5.1] 

Members [1] {
        Member [localhost]:5801 - b2440996-aec2-4c80-8d60-33d8f84dbeab
}

2024-08-04 01:32:14,156 INFO  [.c.i.s.ClientStatisticsService] [main] - Client statistics is enabled with period 5 seconds.
2024-08-04 01:32:14,223 INFO  [o.a.s.e.s.CoordinatorService  ] [pool-5-thread-1] - [localhost]:5801 [seatunnel-651771] [5.1] Loaded event handlers: [org.apache.seatunnel.api.event.LoggingEventHandler@5f232a62]
2024-08-04 01:32:14,226 INFO  [.ClientJobExecutionEnvironment] [main] - add common jar in plugins :[file:/home/apache-seatunnel-2.3.5/plugins/ojdbc8-23.4.0.24.05.jar]
2024-08-04 01:32:14,230 INFO  [.h.i.p.i.PartitionStateManager] [pool-5-thread-1] - [localhost]:5801 [seatunnel-651771] [5.1] Initializing cluster partition table arrangement...
2024-08-04 01:32:14,233 INFO  [o.a.s.c.s.u.ConfigBuilder     ] [main] - Loading config file from path: ./config/v2.streaming.conf.template
2024-08-04 01:32:14,257 INFO  [o.a.s.c.s.u.ConfigShadeUtils  ] [main] - Load config shade spi: [base64]
2024-08-04 01:32:14,298 INFO  [o.a.s.c.s.u.ConfigBuilder     ] [main] - Parsed config file: 
{
    "env" : {
        "parallelism" : 2,
        "job.mode" : "STREAMING",
        "checkpoint.interval" : 2000
    },
    "source" : [
        {
            "base-url" : "jdbc:oracle:thin:@192.168.31.187:1521:ORACLEDB",
            "password" : "dbz",
            "table-names" : [
                "ORACLEDB.c##dbzuser.TEST_STARROCKS"
            ],
            "result_table_name" : "TEST_STARROCKS",
            "database-names" : [
                "ORACLEDB"
            ],
            "schema-names" : [
                "c##dbzuser"
            ],
            "source.reader.close.timeout" : 1200000,
            "plugin_name" : "Oracle-CDC",
            "username" : "c##dbzuser"
        }
    ],
    "sink" : [
        {
            "enable_upsert_delete" : true,
            "password" : "",
            "database" : "test",
            "nodeUrls" : [
                "192.168.31.187:8030"
            ],
            "source_table_name" : [
                "TEST_STARROCKS"
            ],
            "plugin_name" : "StarRocks",
            "table" : "test_seatunnel",
            "username" : "root"
        }
    ]
}

2024-08-04 01:32:14,308 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - Load SeaTunnelSink Plugin from /home/apache-seatunnel-2.3.5/connectors
2024-08-04 01:32:14,312 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - Discovery plugin jar for: PluginIdentifier{engineType='seatunnel', pluginType='source', pluginName='Oracle-CDC'} at: file:/home/apache-seatunnel-2.3.5/connectors/connector-cdc-oracle-2.3.5.jar
2024-08-04 01:32:14,312 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - Discovery plugin jar for: PluginIdentifier{engineType='seatunnel', pluginType='sink', pluginName='StarRocks'} at: file:/home/apache-seatunnel-2.3.5/connectors/connector-starrocks-2.3.5.jar
2024-08-04 01:32:14,318 INFO  [p.MultipleTableJobConfigParser] [main] - start generating all sources.
2024-08-04 01:32:14,566 INFO  [.s.c.s.j.c.AbstractJdbcCatalog] [main] - Catalog Oracle established connection to jdbc:oracle:thin:@192.168.31.187:1521:ORACLEDB
2024-08-04 01:32:18,034 INFO  [.e.s.r.AbstractResourceManager] [hz.main.generic-operation.thread-1] - Init ResourceManager
2024-08-04 01:32:18,034 INFO  [.e.s.r.AbstractResourceManager] [hz.main.generic-operation.thread-1] - initWorker... 
2024-08-04 01:32:18,034 INFO  [.e.s.r.AbstractResourceManager] [hz.main.generic-operation.thread-1] - initWorker live nodes: [[localhost]:5801]
2024-08-04 01:32:18,034 INFO  [.e.s.r.AbstractResourceManager] [hz.main.generic-operation.thread-1] - registerWorker: {[localhost]:5801=WorkerProfile(address=[localhost]:5801, profile=ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=477626368}}, unassignedResource=ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=477626368}}, assignedSlots=[], unassignedSlots=[])}
2024-08-04 01:32:23,002 INFO  [o.a.s.a.t.c.CatalogTableUtil  ] [main] - Get catalog tables, cost time: 8658 ms
2024-08-04 01:32:23,003 INFO  [.s.c.s.j.c.AbstractJdbcCatalog] [main] - Catalog Oracle closing
2024-08-04 01:32:23,008 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel-651771] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is SHUTTING_DOWN
2024-08-04 01:32:23,013 INFO  [c.h.i.s.t.TcpServerConnection ] [hz.main.IO.thread-in-1] - [localhost]:5801 [seatunnel-651771] [5.1] Connection[id=1, /127.0.0.1:5801->/127.0.0.1:52193, qualifier=null, endpoint=[127.0.0.1]:52193, remoteUuid=29d4a84b-c759-4cb5-beca-db775ef7af09, alive=false, connectionType=JVM, planeIndex=-1] closed. Reason: Connection closed by the other side
2024-08-04 01:32:23,013 INFO  [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel-651771] [5.1] Removed connection to endpoint: [localhost]:5801:b2440996-aec2-4c80-8d60-33d8f84dbeab, connection: ClientConnection{alive=false, connectionId=1, channel=NioChannel{/127.0.0.1:52193->localhost/127.0.0.1:5801}, remoteAddress=[localhost]:5801, lastReadTime=2024-08-04 01:32:19.159, lastWriteTime=2024-08-04 01:32:19.159, closedTime=2024-08-04 01:32:23.012, connected server version=5.1}
2024-08-04 01:32:23,013 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel-651771] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is CLIENT_DISCONNECTED
2024-08-04 01:32:23,014 INFO  [c.h.c.i.ClientEndpointManager ] [hz.main.event-4] - [localhost]:5801 [seatunnel-651771] [5.1] Destroying ClientEndpoint{connection=Connection[id=1, /127.0.0.1:5801->/127.0.0.1:52193, qualifier=null, endpoint=[127.0.0.1]:52193, remoteUuid=29d4a84b-c759-4cb5-beca-db775ef7af09, alive=false, connectionType=JVM, planeIndex=-1], clientUuid=29d4a84b-c759-4cb5-beca-db775ef7af09, clientName=hz.client_1, authenticated=true, clientVersion=5.1, creationTime=1722706334138, latest clientAttributes=lastStatisticsCollectionTime=1722706339156,enterprise=false,clientType=JVM,clientVersion=5.1,clusterConnectionTimestamp=1722706334133,clientAddress=127.0.0.1,clientName=hz.client_1,credentials.principal=null,os.committedVirtualMemorySize=16552419328,os.freePhysicalMemorySize=10521702400,os.freeSwapSpaceSize=8438935552,os.maxFileDescriptorCount=262144,os.openFileDescriptorCount=64,os.processCpuTime=6620000000,os.systemLoadAverage=0.04,os.totalPhysicalMemorySize=16472842240,os.totalSwapSpaceSize=8438935552,runtime.availableProcessors=32,runtime.freeMemory=346851512,runtime.maxMemory=477626368,runtime.totalMemory=393216000,runtime.uptime=7052,runtime.usedMemory=46364488, labels=[]}
2024-08-04 01:32:23,016 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel-651771] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is SHUTDOWN
2024-08-04 01:32:23,017 INFO  [s.c.s.s.c.ClientExecuteCommand] [main] - Closed SeaTunnel client......
2024-08-04 01:32:23,017 INFO  [c.h.c.LifecycleService        ] [main] - [localhost]:5801 [seatunnel-651771] [5.1] [localhost]:5801 is SHUTTING_DOWN
2024-08-04 01:32:23,022 INFO  [c.h.i.p.i.MigrationManager    ] [hz.main.cached.thread-2] - [localhost]:5801 [seatunnel-651771] [5.1] Shutdown request of Member [localhost]:5801 - b2440996-aec2-4c80-8d60-33d8f84dbeab this is handled
2024-08-04 01:32:23,024 WARN  [o.a.s.e.s.TaskExecutionService] [pool-2-thread-1] - [localhost]:5801 [seatunnel-651771] [5.1] The Node is not ready yet, Node state PASSIVE,looking forward to the next scheduling
2024-08-04 01:32:23,025 INFO  [c.h.i.i.Node                  ] [main] - [localhost]:5801 [seatunnel-651771] [5.1] Shutting down connection manager...
2024-08-04 01:32:23,027 INFO  [c.h.i.i.Node                  ] [main] - [localhost]:5801 [seatunnel-651771] [5.1] Shutting down node engine...
2024-08-04 01:32:23,032 WARN  [a.s.e.s.s.s.DefaultSlotService] [hz.main.seaTunnel.slotService.thread] - failed send heartbeat to resource manager, will retry later. this address: [localhost]:5801
2024-08-04 01:32:23,033 INFO  [.c.c.DefaultClassLoaderService] [main] - close classloader service
2024-08-04 01:32:23,033 INFO  [o.a.s.e.s.TaskExecutionService] [event-forwarder-0] - [localhost]:5801 [seatunnel-651771] [5.1] Event forward thread interrupted
2024-08-04 01:32:23,043 INFO  [c.h.i.i.NodeExtension         ] [main] - [localhost]:5801 [seatunnel-651771] [5.1] Destroying node NodeExtension.
2024-08-04 01:32:23,043 INFO  [c.h.i.i.Node                  ] [main] - [localhost]:5801 [seatunnel-651771] [5.1] Hazelcast Shutdown is completed in 22 ms.
2024-08-04 01:32:23,043 INFO  [c.h.c.LifecycleService        ] [main] - [localhost]:5801 [seatunnel-651771] [5.1] [localhost]:5801 is SHUTDOWN
2024-08-04 01:32:23,043 INFO  [s.c.s.s.c.ClientExecuteCommand] [main] - Closed HazelcastInstance ......
2024-08-04 01:32:23,043 ERROR [o.a.s.c.s.SeaTunnel           ] [main] - 

===============================================================================

2024-08-04 01:32:23,043 ERROR [o.a.s.c.s.SeaTunnel           ] [main] - Fatal Error, 

2024-08-04 01:32:23,043 ERROR [o.a.s.c.s.SeaTunnel           ] [main] - Please submit bug report in https://github.com/apache/seatunnel/issues

2024-08-04 01:32:23,043 ERROR [o.a.s.c.s.SeaTunnel           ] [main] - Reason:SeaTunnel job executed failed 

2024-08-04 01:32:23,044 ERROR [o.a.s.c.s.SeaTunnel           ] [main] - Exception StackTrace:org.apache.seatunnel.core.starter.exception.CommandExecuteException: SeaTunnel job executed failed
        at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:202)
        at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40)
        at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:34)
Caused by: org.apache.seatunnel.api.table.factory.FactoryException: ErrorCode:[API-06], ErrorDescription:[Factory initialize failed] - Unable to create a source for identifier 'Oracle-CDC'.
        at org.apache.seatunnel.api.table.factory.FactoryUtil.createAndPrepareSource(FactoryUtil.java:100)
        at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.parseSource(MultipleTableJobConfigParser.java:332)
        at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.parse(MultipleTableJobConfigParser.java:188)
        at org.apache.seatunnel.engine.client.job.ClientJobExecutionEnvironment.getLogicalDag(ClientJobExecutionEnvironment.java:88)
        at org.apache.seatunnel.engine.client.job.ClientJobExecutionEnvironment.execute(ClientJobExecutionEnvironment.java:156)
        at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:149)
        ... 2 more
Caused by: org.apache.seatunnel.common.utils.SeaTunnelException: Can not find catalog table with factoryId [Oracle]
        at org.apache.seatunnel.api.table.catalog.CatalogTableUtil.lambda$getCatalogTables$0(CatalogTableUtil.java:129)
        at java.util.Optional.map(Optional.java:215)
        at org.apache.seatunnel.api.table.catalog.CatalogTableUtil.getCatalogTables(CatalogTableUtil.java:116)
        at org.apache.seatunnel.api.table.catalog.CatalogTableUtil.getCatalogTables(CatalogTableUtil.java:96)
        at org.apache.seatunnel.connectors.seatunnel.cdc.oracle.source.OracleIncrementalSourceFactory.lambda$createSource$1(OracleIncrementalSourceFactory.java:106)
        at org.apache.seatunnel.api.table.factory.FactoryUtil.createAndPrepareSource(FactoryUtil.java:112)
        at org.apache.seatunnel.api.table.factory.FactoryUtil.createAndPrepareSource(FactoryUtil.java:73)
        ... 7 more

2024-08-04 01:32:23,044 ERROR [o.a.s.c.s.SeaTunnel           ] [main] - 
===============================================================================

Exception in thread "main" org.apache.seatunnel.core.starter.exception.CommandExecuteException: SeaTunnel job executed failed
        at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:202)
        at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40)
        at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:34)
Caused by: org.apache.seatunnel.api.table.factory.FactoryException: ErrorCode:[API-06], ErrorDescription:[Factory initialize failed] - Unable to create a source for identifier 'Oracle-CDC'.
        at org.apache.seatunnel.api.table.factory.FactoryUtil.createAndPrepareSource(FactoryUtil.java:100)
        at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.parseSource(MultipleTableJobConfigParser.java:332)
        at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.parse(MultipleTableJobConfigParser.java:188)
        at org.apache.seatunnel.engine.client.job.ClientJobExecutionEnvironment.getLogicalDag(ClientJobExecutionEnvironment.java:88)
        at org.apache.seatunnel.engine.client.job.ClientJobExecutionEnvironment.execute(ClientJobExecutionEnvironment.java:156)
        at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:149)
        ... 2 more
Caused by: org.apache.seatunnel.common.utils.SeaTunnelException: Can not find catalog table with factoryId [Oracle]
        at org.apache.seatunnel.api.table.catalog.CatalogTableUtil.lambda$getCatalogTables$0(CatalogTableUtil.java:129)
        at java.util.Optional.map(Optional.java:215)
        at org.apache.seatunnel.api.table.catalog.CatalogTableUtil.getCatalogTables(CatalogTableUtil.java:116)
        at org.apache.seatunnel.api.table.catalog.CatalogTableUtil.getCatalogTables(CatalogTableUtil.java:96)
        at org.apache.seatunnel.connectors.seatunnel.cdc.oracle.source.OracleIncrementalSourceFactory.lambda$createSource$1(OracleIncrementalSourceFactory.java:106)
        at org.apache.seatunnel.api.table.factory.FactoryUtil.createAndPrepareSource(FactoryUtil.java:112)
        at org.apache.seatunnel.api.table.factory.FactoryUtil.createAndPrepareSource(FactoryUtil.java:73)
        ... 7 more

Zeta or Flink or Spark Version

No response

Java or Scala Version

No response

Screenshots

No response

Are you willing to submit PR?

Code of Conduct

cavellpotter commented 1 month ago

oracle用的19c，配置是采用的cdb+pdb方式

cavellpotter commented 1 month ago

image 这里已经找到catelog tables了，但还是closed，最终报错

cavellpotter commented 1 month ago

@hailin0 @Carl-Zhou-CN @Hisoka-X

liunaijie commented 1 month ago

from the log, looks like can't find the table ORACLEDB.c##dbzuser.TEST_STARROCKS i think you can check the table name is right or change to an simple table name to check is there any bug that can't detect this pattern's table name

Carl-Zhou-CN commented 1 month ago

@cavellpotter The pdb needs to merge this pr https://github.com/apache/seatunnel/pull/6647

Carl-Zhou-CN commented 1 month ago

Refer to env { parallelism = 1 job.mode = "STREAMING" checkpoint.interval = 10000 }

source { Oracle-CDC { result_table_name = "fake" connect.timeout.ms = 1200000 source.reader.close.timeout = 1200000 base-url = "jdbc:oracle:thin:@//localhost:1524/ORCLPDB1" username = "system" password = "oracle" database-names = ["ORCLCDB"] schema-names = ["TEST"] table-names = ["ORCLCDB.TEST.T1"] exactly_once = true

debezium { database.pdb.name = "ORCLPDB1" } } }

sink { console { source_table_name="fake" } }

github-actions[bot] commented 1 week ago

This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.