apache / seatunnel

SeaTunnel is a next-generation super high-performance, distributed, massive data integration tool.
https://seatunnel.apache.org/
Apache License 2.0
8.06k stars 1.83k forks source link

使用版本2.3.7使用hudi sink报错 java.lang.NoSuchMethodError: org.apache.avro.Schema.createUnion #8009

Open liangjiaxing2022 opened 1 week ago

liangjiaxing2022 commented 1 week ago

Search before asking

What happened

使用版本2.3.7使用hudi sink报错 java.lang.NoSuchMethodError: org.apache.avro.Schema.createUnion

SeaTunnel Version

2.3.7

SeaTunnel Config

env {
  parallelism = 1
  job.mode = "BATCH"
  checkpoint.interval = 5000
}

source {

 Jdbc {
        url="jdbc:mysql://10.2.5.22:3306/data_dolphin?allowMultiQueries=true&characterEncoding=utf-8"
        driver="com.mysql.cj.jdbc.Driver"
        user = "root"
        password = "Grg@12345"
        result_table_name="t_ds_version"
        query="SELECT id,version  FROM t_ds_version;"
    }

}

transform {
}

sink {
   Hudi {
    source_table_name = "t_ds_version"
    table_dfs_path = "hdfs://10.2.5.22:9001/user/hudi"
    database = "spark_hudi"
    table_name = "t_ds_version"
    table_type = "COPY_ON_WRITE"
    batch_size = 10000
  }
}

Running Command

/apache-seatunnel-2.3.7/bin/seatunnel.sh  --config ./script/hivejdbc.config -m local

Error Exception

java.lang.NoSuchMethodError: org.apache.avro.Schema.createUnion

Zeta or Flink or Spark Version

No response

Java or Scala Version

No response

Screenshots

No response

Are you willing to submit PR?

Code of Conduct