apache / seatunnel

SeaTunnel is a next-generation super high-performance, distributed, massive data integration tool.
https://seatunnel.apache.org/
Apache License 2.0
8.04k stars 1.82k forks source link

[Bug] [SeaTunnelClient] Using SeaTunnelClient submit a using ZetaUDF func job Client throw java.lang.NoClassDefFoundError: org/apache/seatunnel/transform/sql/zeta/ZetaUDF #7859

Open lvxuan263 opened 1 month ago

lvxuan263 commented 1 month ago

Search before asking

What happened

when submit job, client throw exception, the worker doesn't receive job.

SeaTunnel Version

docker 2.3.8 and docker 2.3.7

SeaTunnel Config

# Defining the runtime environment
env {
  parallelism = 1
  job.mode = "BATCH"
}

source{
  FakeSource {
    schema {
      fields {
        image_id = string
        image_file = string
        image_tar = string
        split = string
        description = string
        oritext_version = string
        oritext_description = string
        image_text_similarity = float
        ItoT_start_time = bigint
        ItoT_end_time = bigint
        time = {
          start_time = bigint
          end_time = bigint
        }
      }
    }

    result_table_name = "ori_table"
  }
}

transform {
  Sql {
    source_table_name = "ori_table"
    result_table_name = "result"
    query = """
      select
             uuid() as root_id
     from ori_table
    """
  }
}

sink {
  Console {
    source_table_name = "result"
  }
}

Running Command

./bin/seatunnel.sh  --config ./config/raw_file.config

Error Exception

2024-10-16 11:26:42,684 INFO  [s.c.s.s.c.ClientExecuteCommand] [main] - Closed SeaTunnel client......
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/seatunnel/transform/sql/zeta/ZetaUDF
        at java.lang.ClassLoader.defineClass1(Native Method)
        at java.lang.ClassLoader.defineClass(ClassLoader.java:756)
        at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
        at java.net.URLClassLoader.defineClass(URLClassLoader.java:473)
        at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:405)
        at org.apache.seatunnel.engine.common.loader.SeaTunnelBaseClassLoader.loadClassWithoutExceptionHandling(SeaTunnelBaseClassLoader.java:56)
        at org.apache.seatunnel.engine.common.loader.SeaTunnelChildFirstClassLoader.loadClassWithoutExceptionHandling(SeaTunnelChildFirstClassLoader.java:97)
        at org.apache.seatunnel.engine.common.loader.SeaTunnelBaseClassLoader.loadClass(SeaTunnelBaseClassLoader.java:47)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:348)
        at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:370)
        at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
        at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
        at java.lang.Iterable.forEach(Iterable.java:74)
        at org.apache.seatunnel.transform.sql.zeta.ZetaSQLEngine.init(ZetaSQLEngine.java:83)
        at org.apache.seatunnel.transform.sql.SQLTransform.open(SQLTransform.java:92)
        at org.apache.seatunnel.transform.sql.SQLTransform.tryOpen(SQLTransform.java:101)
        at org.apache.seatunnel.transform.sql.SQLTransform.transformTableSchema(SQLTransform.java:113)
        at org.apache.seatunnel.transform.common.AbstractCatalogSupportTransform.transformCatalogTable(AbstractCatalogSupportTransform.java:90)
        at org.apache.seatunnel.transform.common.AbstractCatalogSupportTransform.getProducedCatalogTable(AbstractCatalogSupportTransform.java:80)
        at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.parseTransform(MultipleTableJobConfigParser.java:477)
        at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.parseTransforms(MultipleTableJobConfigParser.java:405)
        at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.parse(MultipleTableJobConfigParser.java:214)
        at org.apache.seatunnel.engine.client.job.ClientJobExecutionEnvironment.getLogicalDag(ClientJobExecutionEnvironment.java:114)
        at org.apache.seatunnel.engine.client.job.ClientJobExecutionEnvironment.execute(ClientJobExecutionEnvironment.java:182)
        at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:160)
        at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40)
        at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:34)
Caused by: java.lang.ClassNotFoundException: org.apache.seatunnel.transform.sql.zeta.ZetaUDF
        at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
        ... 36 more

Zeta or Flink or Spark Version

zeta 2.3.8

Java or Scala Version

No response

Screenshots

No response

Are you willing to submit PR?

Code of Conduct

github-actions[bot] commented 2 days ago

This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.