apache / seatunnel

SeaTunnel is a next-generation super high-performance, distributed, massive data integration tool.
https://seatunnel.apache.org/
Apache License 2.0
7.82k stars 1.76k forks source link

[Bug] [jdbc] task can not start or stop ,but status is running #7494

Open 13813586515 opened 3 weeks ago

13813586515 commented 3 weeks ago

Search before asking

What happened

提交了jdbc的任务,但是发现每隔一段时间,任务就会被卡主,一直无法执行下去,没有任何报错,执行脚本 seatunnal.sh -j 看状态一直在running状态

SeaTunnel Version

2.3.7

SeaTunnel Config

env{
  parallelism = 4
  job.mode = "BATCH"
  checkpoint.interval = 10000
}

source {
  Jdbc {
    url = "jdbc:oracle:thin:@192.168.98.38:1521:crmls"
    driver = "oracle.jdbc.OracleDriver"
    connection_check_timeout_sec = 100
    user = "abc"
    password = "789"
    table_path = "dbusrpos.sellhead@pos1"
    query = "select *  from dbusrpos.sellhead@pos1 where (SHDATE>=TO_DATE('${last_minute}', 'YYYY/MM/DD HH24:MI:SS') or SHSENDDATE>=TO_DATE('${last_minute}', 'YYYY/MM/DD HH24:MI:SS'))"
    split.size = 10000
  }
}

transform {
  # Sql {
  #   result_table_name="t"
  #   query = "select *,TransformsGBK(STCNAME) as STCNAME from t"
  # }
}

sink {
  Doris {
    source_table_name = "t"
    fenodes = "192.168.0.168:8030,192.168.0.169:8030,192.168.0.170:8030"
    username = root
    password = "222"
    database = "ods_future"
    table = "ods_sellhead"
    # 保证每个文件唯一,拷贝之后文件需要修改此处
    sink.label-prefix = "future-sellhead-jdbc"
    sink.enable-2pc = "true"
    # sink.enable-delete = "true"
    save_mode_create_template="CREATE TABLE IF NOT EXISTS `${database}`.`${table_name}` (${rowtype_fields},dt datetime(0) default current_timestamp(0) on update current_timestamp(0)) ENGINE=OLAP UNIQUE KEY (SHBILLNO) DISTRIBUTED BY HASH (SHBILLNO) BUCKETS 3 PROPERTIES ( \"replication_allocation\" = \"tag.location.default: 3\",\"in_memory\" = \"false\",\"storage_format\" = \"V2\", \"disable_auto_compaction\" = \"false\")"
    doris.config {
      format = "json"
      read_json_by_line = "true"
    }
  }
}

Running Command

集群执行

Error Exception

没有报错,但是就是一直在打印
           Job Progress Information
    ***********************************************
    Job Id                    :  879343235690921985
    Read Count So Far         :                   0
    Write Count So Far        :                   0
    Average Read Count        :                 0/s
    Average Write Count       :                 0/s
    Last Statistic Time       : 2024-08-23 20:40:59
    Current Statistic Time    : 2024-08-23 20:41:08
    ***********************************************

Zeta or Flink or Spark Version

zeta

Java or Scala Version

1.8

Screenshots

image image image

Are you willing to submit PR?

Code of Conduct

hailin0 commented 3 weeks ago

You need to use jstack to view the task thread stack