search

apache / seatunnel

SeaTunnel is a next-generation super high-performance, distributed, massive data integration tool.
https://seatunnel.apache.org/
Apache License 2.0
7.99k stars 1.81k forks source link

[Bug] [seatunnel-engine-server] when using rest-api hazelcast/rest/maps/running-job/:job-id，return exception：java.lang.ClassNotFoundException: org.apache.seatunnel.connectors.seatunnel.kafka.source.KafkaSource #6192

Closed NighterCoder closed 7 months ago

NighterCoder commented 9 months ago

Search before asking

What happened

when using rest-api hazelcast/rest/maps/running-job/:job-id， the task ran normally，but monitor http command return exception：java.lang.ClassNotFoundException: org.apache.seatunnel.connectors.seatunnel.kafka.source.KafkaSource

SeaTunnel Version

org.apache.seatunnel seatunnel 2.3.3-SNAPSHOT

SeaTunnel Config

env {
"job.mode"=streaming
"execution.parallelism"=1
}
source {
kafka {
    format=json
    "start_mode"=latest
    "kafka.config" {
        "max.poll.records"="500"
        "client.id"="client_1"
    }
    "result_table_name"=task1SourceTable
    topic=testst00
    schema {
        fields {
            name=string
            age=int
        }
    }
    "bootstrap.servers"="xxxx"
}
}
transform {
}
sink {
kafka {
    format=text
    "source_table_name"=task1SinkTable
    topic=testst01
    "bootstrap.servers"="xxxx"
}
}

Running Command

$seatunnel_home/bin/seatunnel.sh --config seatunnel_kafka.conf

Error Exception

An error occurred while handling request HttpCommand [HTTP_GET]{uri='/hazelcast/rest/maps/running-jobs'}AbstractTextCommand[HTTP_GET]{requestId=0}
com.hazelcast.nio.serialization.HazelcastSerializationException: java.lang.ClassNotFoundException: org.apache.seatunnel.connectors.seatunnel.kafka.source.KafkaSource
        at com.hazelcast.internal.serialization.impl.defaultserializers.JavaDefaultSerializers$JavaSerializer.read(JavaDefaultSerializers.java:96) ~[seatunnel-starter.jar:2.3.3-SNAPSHOT]
        at com.hazelcast.internal.serialization.impl.defaultserializers.JavaDefaultSerializers$JavaSerializer.read(JavaDefaultSerializers.java:85) ~[seatunnel-starter.jar:2.3.3-SNAPSHOT]
        at com.hazelcast.internal.serialization.impl.StreamSerializerAdapter.read(StreamSerializerAdapter.java:44) ~[seatunnel-starter.jar:2.3.3-SNAPSHOT]
        at com.hazelcast.internal.serialization.impl.AbstractSerializationService.readObject(AbstractSerializationService.java:349) ~[seatunnel-starter.jar:2.3.3-SNAPSHOT]
        at com.hazelcast.internal.serialization.impl.ByteArrayObjectDataInput.readObject(ByteArrayObjectDataInput.java:600) ~[seatunnel-starter.jar:2.3.3-SNAPSHOT]
        at org.apache.seatunnel.engine.core.dag.logical.LogicalVertex.readData(LogicalVertex.java:99) ~[seatunnel-starter.jar:2.3.3-SNAPSHOT]
        at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.readInternal(DataSerializableSerializer.java:160) ~[seatunnel-starter.jar:2.3.3-SNAPSHOT]
        at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.read(DataSerializableSerializer.java:106) ~[seatunnel-starter.jar:2.3.3-SNAPSHOT]
        at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.read(DataSerializableSerializer.java:51) ~[seatunnel-starter.jar:2.3.3-SNAPSHOT]
        at com.hazelcast.internal.serialization.impl.StreamSerializerAdapter.read(StreamSerializerAdapter.java:44) ~[seatunnel-starter.jar:2.3.3-SNAPSHOT]
        at com.hazelcast.internal.serialization.impl.AbstractSerializationService.readObject(AbstractSerializationService.java:349) ~[seatunnel-starter.jar:2.3.3-SNAPSHOT]
        at com.hazelcast.internal.serialization.impl.ByteArrayObjectDataInput.readObject(ByteArrayObjectDataInput.java:600) ~[seatunnel-starter.jar:2.3.3-SNAPSHOT]
        at org.apache.seatunnel.engine.core.dag.logical.LogicalDag.readData(LogicalDag.java:154) ~[seatunnel-starter.jar:2.3.3-SNAPSHOT]
        at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.readInternal(DataSerializableSerializer.java:160) ~[seatunnel-starter.jar:2.3.3-SNAPSHOT]
        at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.read(DataSerializableSerializer.java:106) ~[seatunnel-starter.jar:2.3.3-SNAPSHOT]
        at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.read(DataSerializableSerializer.java:51) ~[seatunnel-starter.jar:2.3.3-SNAPSHOT]
        at com.hazelcast.internal.serialization.impl.StreamSerializerAdapter.read(StreamSerializerAdapter.java:44) ~[seatunnel-starter.jar:2.3.3-SNAPSHOT]
        at com.hazelcast.internal.serialization.impl.AbstractSerializationService.toObject(AbstractSerializationService.java:268) ~[seatunnel-starter.jar:2.3.3-SNAPSHOT]
        at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708) ~[?:1.8.0_372]
        at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:1.8.0_372]
        at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:577) ~[?:1.8.0_372]
        at org.apache.seatunnel.engine.server.rest.RestHttpGetCommandProcessor.handleRunningJobsInfo(RestHttpGetCommandProcessor.java:162) ~[seatunnel-starter.jar:2.3.3-SNAPSHOT]
        at org.apache.seatunnel.engine.server.rest.RestHttpGetCommandProcessor.handle(RestHttpGetCommandProcessor.java:89) ~[seatunnel-starter.jar:2.3.3-SNAPSHOT]
        at org.apache.seatunnel.engine.server.rest.RestHttpGetCommandProcessor.handle(RestHttpGetCommandProcessor.java:61) ~[seatunnel-starter.jar:2.3.3-SNAPSHOT]
        at com.hazelcast.internal.ascii.TextCommandServiceImpl$CommandExecutor.run(TextCommandServiceImpl.java:402) ~[seatunnel-starter.jar:2.3.3-SNAPSHOT]
        at com.hazelcast.internal.util.executor.CachedExecutorServiceDelegate$Worker.run(CachedExecutorServiceDelegate.java:217) ~[seatunnel-starter.jar:2.3.3-SNAPSHOT]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_372]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_372]
        at java.lang.Thread.run(Thread.java:750) [?:1.8.0_372]
        at com.hazelcast.internal.util.executor.HazelcastManagedThread.executeRun(HazelcastManagedThread.java:76) ~[seatunnel-starter.jar:2.3.3-SNAPSHOT]
        at com.hazelcast.internal.util.executor.HazelcastManagedThread.run(HazelcastManagedThread.java:102) ~[seatunnel-starter.jar:2.3.3-SNAPSHOT]
Caused by: java.lang.ClassNotFoundException: org.apache.seatunnel.connectors.seatunnel.kafka.source.KafkaSource
        at java.net.URLClassLoader.findClass(URLClassLoader.java:387) ~[?:1.8.0_372]
        at java.lang.ClassLoader.loadClass(ClassLoader.java:418) ~[?:1.8.0_372]
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) ~[?:1.8.0_372]
        at java.lang.ClassLoader.loadClass(ClassLoader.java:351) ~[?:1.8.0_372]
        at com.hazelcast.internal.nio.ClassLoaderUtil.tryLoadClass(ClassLoaderUtil.java:301) ~[seatunnel-starter.jar:2.3.3-SNAPSHOT]
        at com.hazelcast.internal.nio.ClassLoaderUtil.loadClass(ClassLoaderUtil.java:259) ~[seatunnel-starter.jar:2.3.3-SNAPSHOT]
        at com.hazelcast.internal.nio.IOUtil$ClassLoaderAwareObjectInputStream.resolveClass(IOUtil.java:867) ~[seatunnel-starter.jar:2.3.3-SNAPSHOT]
        at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1987) ~[?:1.8.0_372]
        at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1851) ~[?:1.8.0_372]
        at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2185) ~[?:1.8.0_372]
        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1668) ~[?:1.8.0_372]
        at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2430) ~[?:1.8.0_372]
        at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2354) ~[?:1.8.0_372]
        at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2212) ~[?:1.8.0_372]
        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1668) ~[?:1.8.0_372]
        at java.io.ObjectInputStream.readObject(ObjectInputStream.java:502) ~[?:1.8.0_372]
        at java.io.ObjectInputStream.readObject(ObjectInputStream.java:460) ~[?:1.8.0_372]
        at com.hazelcast.internal.serialization.impl.defaultserializers.JavaDefaultSerializers$JavaSerializer.read(JavaDefaultSerializers.java:92) ~[seatunnel-starter.jar:2.3.3-SNAPSHOT]
        ... 37 more

Zeta or Flink or Spark Version

Zeta

Java or Scala Version

Java 1.8

Screenshots

No response

Are you willing to submit PR?

Code of Conduct

joswlv commented 9 months ago

Most likely, you didn't install the libs.

Before packaging the source code, run {seatunnel_source_code}/bin/install-plugin.sh (this will download the relevant lib jars under a dir called connectors) and then run mvn clean package -DskipTests -T 1C to deploy the results.

The packaged result can be used by deploying the tar under the seatunnel-dist/target dir to your server.

NighterCoder commented 9 months ago

WechatIMG513 WechatIMG514 I installed the libs， above are lib dir and seatunnel dir，is any package missing？

NighterCoder commented 9 months ago

upgrade seatunnel version to latest version 2.3.4-SNAPSHOT，this problem has solved

liugddx commented 9 months ago

use 2.3.4 and try again.

github-actions[bot] commented 7 months ago

This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

github-actions[bot] commented 7 months ago

This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future.