search

DTStack / chunjun

A data integration framework
https://dtstack.github.io/chunjun/
Apache License 2.0
3.98k stars 1.69k forks source link

是否支持restful api创建任务？或者通过flink restful API是否可以实现chunjun任务的提交？ #1409

Open 382035656 opened 1 year ago

382035656 commented 1 year ago

Search before asking

Description

纯钧本身是否可以做到restful api创建任务？ 我自己尝试使用flink RestClusterClient去创建任务，但在构建任务时JobGraph总是会报ClassNotFound

`D:\jdk\jdk1.8.0_60\bin\java.exe -agentlib:jdwp=transport=dt_socket,address=127.0.0.1:59771,suspend=y,server=n -javaagent:C:\Users\Administrator\AppData\Local\JetBrains\IntelliJIdea2021.2\captureAgent\debugger-agent.jar -Dfile.encoding=UTF-8 -classpath "D:\jdk\jdk1.8.0_60\jre\lib\charsets.jar;D:\jdk\jdk1.8.0_60\jre\lib\deploy.jar;D:\jdk\jdk1.8.0_60\jre\lib\ext\access-bridge-64.jar;D:\jdk\jdk1.8.0_60\jre\lib\ext\cldrdata.jar;D:\jdk\jdk1.8.0_60\jre\lib\ext\dnsns.jar;D:\jdk\jdk1.8.0_60\jre\lib\ext\jaccess.jar;D:\jdk\jdk1.8.0_60\jre\lib\ext\jfxrt.jar;D:\jdk\jdk1.8.0_60\jre\lib\ext\localedata.jar;D:\jdk\jdk1.8.0_60\jre\lib\ext\nashorn.jar;D:\jdk\jdk1.8.0_60\jre\lib\ext\sunec.jar;D:\jdk\jdk1.8.0_60\jre\lib\ext\sunjce_provider.jar;D:\jdk\jdk1.8.0_60\jre\lib\ext\sunmscapi.jar;D:\jdk\jdk1.8.0_60\jre\lib\ext\sunpkcs11.jar;D:\jdk\jdk1.8.0_60\jre\lib\ext\zipfs.jar;D:\jdk\jdk1.8.0_60\jre\lib\javaws.jar;D:\jdk\jdk1.8.0_60\jre\lib\jce.jar;D:\jdk\jdk1.8.0_60\jre\lib\jfr.jar;D:\jdk\jdk1.8.0_60\jre\lib\jfxswt.jar;D:\jdk\jdk1.8.0_60\jre\lib\jsse.jar;D:\jdk\jdk1.8.0_60\jre\lib\management-agent.jar;D:\jdk\jdk1.8.0_60\jre\lib\plugin.jar;D:\jdk\jdk1.8.0_60\jre\lib\resources.jar;D:\jdk\jdk1.8.0_60\jre\lib\rt.jar;F:\IDEA_Code\lzhaoxia\flink_demo\target\test-classes;F:\IDEA_Code\lzhaoxia\flink_demo\target\classes;C:\Users\Administrator.m2\repository\org\springframework\boot\spring-boot-starter\2.7.5\spring-boot-starter-2.7.5.jar;C:\Users\Administrator.m2\repository\org\springframework\boot\spring-boot\2.7.5\spring-boot-2.7.5.jar;C:\Users\Administrator.m2\repository\org\springframework\spring-context\5.3.23\spring-context-5.3.23.jar;C:\Users\Administrator.m2\repository\org\springframework\spring-aop\5.3.23\spring-aop-5.3.23.jar;C:\Users\Administrator.m2\repository\org\springframework\spring-beans\5.3.23\spring-beans-5.3.23.jar;C:\Users\Administrator.m2\repository\org\springframework\spring-expression\5.3.23\spring-expression-5.3.23.jar;C:\Users\Administrator.m2\repository\org\springframework\boot\spring-boot-autoconfigure\2.7.5\spring-boot-autoconfigure-2.7.5.jar;C:\Users\Administrator.m2\repository\org\springframework\boot\spring-boot-starter-logging\2.7.5\spring-boot-starter-logging-2.7.5.jar;C:\Users\Administrator.m2\repository\ch\qos\logback\logback-classic\1.2.11\logback-classic-1.2.11.jar;C:\Users\Administrator.m2\repository\ch\qos\logback\logback-core\1.2.11\logback-core-1.2.11.jar;C:\Users\Administrator.m2\repository\org\apache\logging\log4j\log4j-to-slf4j\2.17.2\log4j-to-slf4j-2.17.2.jar;C:\Users\Administrator.m2\repository\org\apache\logging\log4j\log4j-api\2.17.2\log4j-api-2.17.2.jar;C:\Users\Administrator.m2\repository\org\slf4j\jul-to-slf4j\1.7.36\jul-to-slf4j-1.7.36.jar;C:\Users\Administrator.m2\repository\jakarta\annotation\jakarta.annotation-api\1.3.5\jakarta.annotation-api-1.3.5.jar;C:\Users\Administrator.m2\repository\org\springframework\spring-core\5.3.23\spring-core-5.3.23.jar;C:\Users\Administrator.m2\repository\org\springframework\spring-jcl\5.3.23\spring-jcl-5.3.23.jar;C:\Users\Administrator.m2\repository\org\yaml\snakeyaml\1.30\snakeyaml-1.30.jar;C:\Users\Administrator.m2\repository\org\apache\flink\flink-clients_2.12\1.12.7\flink-clients_2.12-1.12.7.jar;C:\Users\Administrator.m2\repository\org\apache\flink\flink-core\1.12.7\flink-core-1.12.7.jar;C:\Users\Administrator.m2\repository\org\apache\flink\flink-annotations\1.12.7\flink-annotations-1.12.7.jar;C:\Users\Administrator.m2\repository\org\apache\flink\flink-metrics-core\1.12.7\flink-metrics-core-1.12.7.jar;C:\Users\Administrator.m2\repository\org\apache\flink\flink-shaded-asm-7\7.1-12.0\flink-shaded-asm-7-7.1-12.0.jar;C:\Users\Administrator.m2\repository\org\apache\commons\commons-lang3\3.12.0\commons-lang3-3.12.0.jar;C:\Users\Administrator.m2\repository\com\esotericsoftware\kryo\kryo\2.24.0\kryo-2.24.0.jar;C:\Users\Administrator.m2\repository\com\esotericsoftware\minlog\minlog\1.2\minlog-1.2.jar;C:\Users\Administrator.m2\repository\commons-collections\commons-collections\3.2.2\commons-collections-3.2.2.jar;C:\Users\Administrator.m2\repository\org\apache\commons\commons-compress\1.20\commons-compress-1.20.jar;C:\Users\Administrator.m2\repository\org\apache\flink\flink-shaded-guava\18.0-12.0\flink-shaded-guava-18.0-12.0.jar;C:\Users\Administrator.m2\repository\org\apache\flink\flink-runtime_2.12\1.12.7\flink-runtime_2.12-1.12.7.jar;C:\Users\Administrator.m2\repository\org\apache\flink\flink-queryable-state-client-java\1.12.7\flink-queryable-state-client-java-1.12.7.jar;C:\Users\Administrator.m2\repository\org\apache\flink\flink-hadoop-fs\1.12.7\flink-hadoop-fs-1.12.7.jar;C:\Users\Administrator.m2\repository\commons-io\commons-io\2.8.0\commons-io-2.8.0.jar;C:\Users\Administrator.m2\repository\org\apache\flink\flink-shaded-netty\4.1.49.Final-12.0\flink-shaded-netty-4.1.49.Final-12.0.jar;C:\Users\Administrator.m2\repository\org\apache\flink\flink-shaded-jackson\2.10.1-12.0\flink-shaded-jackson-2.10.1-12.0.jar;C:\Users\Administrator.m2\repository\org\apache\flink\flink-shaded-zookeeper-3\3.4.14-12.0\flink-shaded-zookeeper-3-3.4.14-12.0.jar;C:\Users\Administrator.m2\repository\org\javassist\javassist\3.24.0-GA\javassist-3.24.0-GA.jar;C:\Users\Administrator.m2\repository\org\scala-lang\scala-library\2.12.7\scala-library-2.12.7.jar;C:\Users\Administrator.m2\repository\com\typesafe\akka\akka-actor_2.12\2.5.21\akka-actor_2.12-2.5.21.jar;C:\Users\Administrator.m2\repository\com\typesafe\config\1.3.3\config-1.3.3.jar;C:\Users\Administrator.m2\repository\org\scala-lang\modules\scala-java8-compat_2.12\0.8.0\scala-java8-compat_2.12-0.8.0.jar;C:\Users\Administrator.m2\repository\com\typesafe\akka\akka-stream_2.12\2.5.21\akka-stream_2.12-2.5.21.jar;C:\Users\Administrator.m2\repository\org\reactivestreams\reactive-streams\1.0.4\reactive-streams-1.0.4.jar;C:\Users\Administrator.m2\repository\com\typesafe\ssl-config-core_2.12\0.3.7\ssl-config-core_2.12-0.3.7.jar;C:\Users\Administrator.m2\repository\org\scala-lang\modules\scala-parser-combinators_2.12\1.1.1\scala-parser-combinators_2.12-1.1.1.jar;C:\Users\Administrator.m2\repository\com\typesafe\akka\akka-protobuf_2.12\2.5.21\akka-protobuf_2.12-2.5.21.jar;C:\Users\Administrator.m2\repository\com\typesafe\akka\akka-slf4j_2.12\2.5.21\akka-slf4j_2.12-2.5.21.jar;C:\Users\Administrator.m2\repository\org\clapper\grizzled-slf4j_2.12\1.3.2\grizzled-slf4j_2.12-1.3.2.jar;C:\Users\Administrator.m2\repository\com\github\scopt\scopt_2.12\3.5.0\scopt_2.12-3.5.0.jar;C:\Users\Administrator.m2\repository\org\xerial\snappy\snappy-java\1.1.8.3\snappy-java-1.1.8.3.jar;C:\Users\Administrator.m2\repository\com\twitter\chill_2.12\0.7.6\chill_2.12-0.7.6.jar;C:\Users\Administrator.m2\repository\com\twitter\chill-java\0.7.6\chill-java-0.7.6.jar;C:\Users\Administrator.m2\repository\org\lz4\lz4-java\1.6.0\lz4-java-1.6.0.jar;C:\Users\Administrator.m2\repository\org\apache\flink\flink-optimizer_2.12\1.12.7\flink-optimizer_2.12-1.12.7.jar;C:\Users\Administrator.m2\repository\org\apache\flink\flink-java\1.12.7\flink-java-1.12.7.jar;C:\Users\Administrator.m2\repository\org\apache\commons\commons-math3\3.5\commons-math3-3.5.jar;C:\Users\Administrator.m2\repository\commons-cli\commons-cli\1.3.1\commons-cli-1.3.1.jar;C:\Users\Administrator.m2\repository\org\apache\flink\flink-streaming-java_2.12\1.12.7\flink-streaming-java_2.12-1.12.7.jar;C:\Users\Administrator.m2\repository\org\apache\flink\flink-file-sink-common\1.12.7\flink-file-sink-common-1.12.7.jar;C:\Users\Administrator.m2\repository\org\slf4j\slf4j-api\1.7.36\slf4j-api-1.7.36.jar;C:\Users\Administrator.m2\repository\com\google\code\findbugs\jsr305\1.3.9\jsr305-1.3.9.jar;C:\Users\Administrator.m2\repository\org\apache\flink\force-shading\1.12.7\force-shading-1.12.7.jar;C:\Users\Administrator.m2\repository\org\projectlombok\lombok\1.18.24\lombok-1.18.24.jar;C:\Users\Administrator.m2\repository\org\springframework\boot\spring-boot-starter-test\2.7.5\spring-boot-starter-test-2.7.5.jar;C:\Users\Administrator.m2\repository\org\springframework\boot\spring-boot-test\2.7.5\spring-boot-test-2.7.5.jar;C:\Users\Administrator.m2\repository\org\springframework\boot\spring-boot-test-autoconfigure\2.7.5\spring-boot-test-autoconfigure-2.7.5.jar;C:\Users\Administrator.m2\repository\com\jayway\jsonpath\json-path\2.7.0\json-path-2.7.0.jar;C:\Users\Administrator.m2\repository\net\minidev\json-smart\2.4.8\json-smart-2.4.8.jar;C:\Users\Administrator.m2\repository\net\minidev\accessors-smart\2.4.8\accessors-smart-2.4.8.jar;C:\Users\Administrator.m2\repository\org\ow2\asm\asm\9.1\asm-9.1.jar;C:\Users\Administrator.m2\repository\jakarta\xml\bind\jakarta.xml.bind-api\2.3.3\jakarta.xml.bind-api-2.3.3.jar;C:\Users\Administrator.m2\repository\jakarta\activation\jakarta.activation-api\1.2.2\jakarta.activation-api-1.2.2.jar;C:\Users\Administrator.m2\repository\org\assertj\assertj-core\3.22.0\assertj-core-3.22.0.jar;C:\Users\Administrator.m2\repository\org\hamcrest\hamcrest\2.2\hamcrest-2.2.jar;C:\Users\Administrator.m2\repository\org\junit\jupiter\junit-jupiter\5.8.2\junit-jupiter-5.8.2.jar;C:\Users\Administrator.m2\repository\org\junit\jupiter\junit-jupiter-api\5.8.2\junit-jupiter-api-5.8.2.jar;C:\Users\Administrator.m2\repository\org\opentest4j\opentest4j\1.2.0\opentest4j-1.2.0.jar;C:\Users\Administrator.m2\repository\org\junit\platform\junit-platform-commons\1.8.2\junit-platform-commons-1.8.2.jar;C:\Users\Administrator.m2\repository\org\apiguardian\apiguardian-api\1.1.2\apiguardian-api-1.1.2.jar;C:\Users\Administrator.m2\repository\org\junit\jupiter\junit-jupiter-params\5.8.2\junit-jupiter-params-5.8.2.jar;C:\Users\Administrator.m2\repository\org\junit\jupiter\junit-jupiter-engine\5.8.2\junit-jupiter-engine-5.8.2.jar;C:\Users\Administrator.m2\repository\org\junit\platform\junit-platform-engine\1.8.2\junit-platform-engine-1.8.2.jar;C:\Users\Administrator.m2\repository\org\mockito\mockito-core\4.5.1\mockito-core-4.5.1.jar;C:\Users\Administrator.m2\repository\net\bytebuddy\byte-buddy\1.12.18\byte-buddy-1.12.18.jar;C:\Users\Administrator.m2\repository\net\bytebuddy\byte-buddy-agent\1.12.18\byte-buddy-agent-1.12.18.jar;C:\Users\Administrator.m2\repository\org\objenesis\objenesis\3.2\objenesis-3.2.jar;C:\Users\Administrator.m2\repository\org\mockito\mockito-junit-jupiter\4.5.1\mockito-junit-jupiter-4.5.1.jar;C:\Users\Administrator.m2\repository\org\skyscreamer\jsonassert\1.5.1\jsonassert-1.5.1.jar;C:\Users\Administrator.m2\repository\com\vaadin\external\google\android-json\0.0.20131108.vaadin1\android-json-0.0.20131108.vaadin1.jar;C:\Users\Administrator.m2\repository\org\springframework\spring-test\5.3.23\spring-test-5.3.23.jar;C:\Users\Administrator.m2\repository\org\xmlunit\xmlunit-core\2.9.0\xmlunit-core-2.9.0.jar;C:\Users\Administrator.m2\repository\javax\servlet\servlet-api\2.3\servlet-api-2.3.jar;C:\Users\Administrator.m2\repository\com\squareup\okhttp3\okhttp\4.9.0\okhttp-4.9.0.jar;C:\Users\Administrator.m2\repository\com\squareup\okio\okio\2.8.0\okio-2.8.0.jar;C:\Users\Administrator.m2\repository\org\jetbrains\kotlin\kotlin-stdlib-common\1.6.21\kotlin-stdlib-common-1.6.21.jar;C:\Users\Administrator.m2\repository\org\jetbrains\kotlin\kotlin-stdlib\1.6.21\kotlin-stdlib-1.6.21.jar;C:\Users\Administrator.m2\repository\org\jetbrains\annotations\13.0\annotations-13.0.jar;C:\Users\Administrator.m2\repository\com\squareup\okhttp3\logging-interceptor\4.9.0\logging-interceptor-4.9.0.jar;C:\Users\Administrator.m2\repository\org\jetbrains\kotlin\kotlin-stdlib-jdk8\1.6.21\kotlin-stdlib-jdk8-1.6.21.jar;C:\Users\Administrator.m2\repository\org\jetbrains\kotlin\kotlin-stdlib-jdk7\1.6.21\kotlin-stdlib-jdk7-1.6.21.jar;C:\Users\Administrator.m2\repository\com\alibaba\fastjson\1.2.74\fastjson-1.2.74.jar;C:\Users\Administrator.m2\repository\cn\hutool\hutool-all\5.7.20\hutool-all-5.7.20.jar;D:\IDEA\IntelliJ IDEA 2021.2.2\lib\idea_rt.jar" com.lizhaoxia.flink_demo.FlinkDemoApplicationTests Connected to the target VM, address: '127.0.0.1:59771', transport: 'socket' 14:32:23.161 [main] INFO org.apache.flink.configuration.Configuration - Config uses fallback configuration key 'jobmanager.rpc.address' instead of key 'rest.address' 14:32:23.838 [main] DEBUG org.apache.flink.shaded.netty4.io.netty.util.internal.logging.InternalLoggerFactory - Using SLF4J as the default logging framework 14:32:23.853 [main] DEBUG org.apache.flink.shaded.netty4.io.netty.channel.MultithreadEventLoopGroup - -Dio.netty.eventLoopThreads: 16 14:32:23.902 [main] DEBUG org.apache.flink.shaded.netty4.io.netty.util.internal.InternalThreadLocalMap - -Dio.netty.threadLocalMap.stringBuilder.initialSize: 1024 14:32:23.903 [main] DEBUG org.apache.flink.shaded.netty4.io.netty.util.internal.InternalThreadLocalMap - -Dio.netty.threadLocalMap.stringBuilder.maxSize: 4096 14:32:23.920 [main] DEBUG org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop - -Dio.netty.noKeySetOptimization: false 14:32:23.920 [main] DEBUG org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop - -Dio.netty.selectorAutoRebuildThreshold: 512 14:32:23.958 [main] DEBUG org.apache.flink.shaded.netty4.io.netty.util.internal.PlatformDependent - Platform: Windows 14:32:23.961 [main] DEBUG org.apache.flink.shaded.netty4.io.netty.util.internal.PlatformDependent0 - -Dio.netty.noUnsafe: false 14:32:23.962 [main] DEBUG org.apache.flink.shaded.netty4.io.netty.util.internal.PlatformDependent0 - Java version: 8 14:32:23.965 [main] DEBUG org.apache.flink.shaded.netty4.io.netty.util.internal.PlatformDependent0 - sun.misc.Unsafe.theUnsafe: available 14:32:23.966 [main] DEBUG org.apache.flink.shaded.netty4.io.netty.util.internal.PlatformDependent0 - sun.misc.Unsafe.copyMemory: available 14:32:23.968 [main] DEBUG org.apache.flink.shaded.netty4.io.netty.util.internal.PlatformDependent0 - java.nio.Buffer.address: available 14:32:23.970 [main] DEBUG org.apache.flink.shaded.netty4.io.netty.util.internal.PlatformDependent0 - direct buffer constructor: available 14:32:23.972 [main] DEBUG org.apache.flink.shaded.netty4.io.netty.util.internal.PlatformDependent0 - java.nio.Bits.unaligned: available, true 14:32:23.973 [main] DEBUG org.apache.flink.shaded.netty4.io.netty.util.internal.PlatformDependent0 - jdk.internal.misc.Unsafe.allocateUninitializedArray(int): unavailable prior to Java9 14:32:23.973 [main] DEBUG org.apache.flink.shaded.netty4.io.netty.util.internal.PlatformDependent0 - java.nio.DirectByteBuffer.(long, int): available 14:32:23.973 [main] DEBUG org.apache.flink.shaded.netty4.io.netty.util.internal.PlatformDependent - sun.misc.Unsafe: available 14:32:23.974 [main] DEBUG org.apache.flink.shaded.netty4.io.netty.util.internal.PlatformDependent - -Dio.netty.tmpdir: C:\Users\ADMINI~1\AppData\Local\Temp (java.io.tmpdir) 14:32:23.974 [main] DEBUG org.apache.flink.shaded.netty4.io.netty.util.internal.PlatformDependent - -Dio.netty.bitMode: 64 (sun.arch.data.model) 14:32:23.978 [main] DEBUG org.apache.flink.shaded.netty4.io.netty.util.internal.PlatformDependent - -Dio.netty.maxDirectMemory: 3791650816 bytes 14:32:23.978 [main] DEBUG org.apache.flink.shaded.netty4.io.netty.util.internal.PlatformDependent - -Dio.netty.uninitializedArrayAllocationThreshold: -1 14:32:23.980 [main] DEBUG org.apache.flink.shaded.netty4.io.netty.util.internal.CleanerJava6 - java.nio.ByteBuffer.cleaner(): available 14:32:23.981 [main] DEBUG org.apache.flink.shaded.netty4.io.netty.util.internal.PlatformDependent - -Dio.netty.noPreferDirect: false 14:32:24.016 [main] DEBUG org.apache.flink.shaded.netty4.io.netty.util.internal.PlatformDependent - org.jctools-core.MpscChunkedArrayQueue: available 14:32:24.090 [main] DEBUG org.apache.flink.runtime.rest.RestClient - Rest client endpoint started. 14:32:45.668 [main] INFO com.dtstack.chunjun.Main - ------------program params------------------------- 14:32:45.670 [main] INFO com.dtstack.chunjun.Main - -flinkLibDir 14:32:45.670 [main] INFO com.dtstack.chunjun.Main - F:\IDEA_Code\szhn\demo\chunjun\zip\flink-1.12.7\lib 14:32:45.670 [main] INFO com.dtstack.chunjun.Main - -p 14:32:45.670 [main] INFO com.dtstack.chunjun.Main -
14:32:45.670 [main] INFO com.dtstack.chunjun.Main - -job 14:32:45.670 [main] INFO com.dtstack.chunjun.Main - %7B%0D%0A++++%22job%22%3A+%7B%0D%0A++++++++%22content%22%3A+%5B%7B%0D%0A++++++++++++++++%22reader%22%3A+%7B%0D%0A++++++++++++++++++++%22name%22%3A+%22mysqlreader%22%2C%0D%0A++++++++++++++++++++%22parameter%22%3A+%7B%0D%0A++++++++++++++++++++++++%22column%22%3A+%5B%7B%0D%0A++++++++++++++++++++++++++++++++%22name%22%3A+%22id%22%2C%0D%0A++++++++++++++++++++++++++++++++%22type%22%3A+%22int%22%0D%0A++++++++++++++++++++++++++++%7D%2C+%7B%0D%0A++++++++++++++++++++++++++++++++%22name%22%3A+%22name%22%2C%0D%0A++++++++++++++++++++++++++++++++%22type%22%3A+%22string%22%0D%0A++++++++++++++++++++++++++++%7D%2C%7B%0D%0A%09%09%09%09%09%09%09%09%22name%22%3A+%22update_time%22%2C%0D%0A%09%09%09%09%09%09%09%09%22type%22%3A+%22datetime%22%0D%0A%09%09%09%09%09%09%09%7D%0D%0A++++++++++++++++++++++++%5D%2C%0D%0A++++++++++++++++++++++++%22connection%22%3A+%5B%7B%0D%0A++++++++++++++++++++++++++++++++%22jdbcUrl%22%3A+%5B%22jdbc%3Amysql%3A%2F%2F192.168.21.71%3A3306%2Ftest%3FuseSSL%3Dfalse%22%5D%2C%0D%0A++++++++++++++++++++++++++++++++%22table%22%3A+%5B%22test%22%5D%0D%0A++++++++++++++++++++++++++++%7D%0D%0A++++++++++++++++++++++++%5D%2C%0D%0A++++++++++++++++++++++++%22customSql%22%3A+%22%22%2C%0D%0A++++++++++++++++++++++++%22increColumn%22%3A+%22update_time%22%2C%0D%0A++++++++++++++++++++++++%22password%22%3A+%22root%22%2C%0D%0A++++++++++++++++++++++++%22polling%22%3A+true%2C%0D%0A++++++++++++++++++++++++%22pollingInterval%22%3A+3000%2C%0D%0A++++++++++++++++++++++++%22queryTimeOut%22%3A+1000%2C%0D%0A++++++++++++++++++++++++%22splitPk%22%3A+%22update_time%22%2C%0D%0A++++++++++++++++++++++++%22startLocation%22%3A+%222022-11-21+12%3A32%3A05%22%2C%0D%0A++++++++++++++++++++++++%22username%22%3A+%22root%22%2C%0D%0A++++++++++++++++++++++++%22where%22%3A+%22%22%0D%0A++++++++++++++++++++%7D%0D%0A++++++++++++++++%7D%2C%0D%0A++++++++++++++++%22writer%22%3A+%7B%0D%0A++++++++++++++++++++%22name%22%3A+%22mysqlwriter%22%2C%0D%0A++++++++++++++++++++%22parameter%22%3A+%7B%0D%0A++++++++++++++++++++++++%22column%22%3A+%5B%7B%0D%0A++++++++++++++++++++++++++++++++%22name%22%3A+%22id%22%2C%0D%0A++++++++++++++++++++++++++++++++%22type%22%3A+%22int%22%0D%0A++++++++++++++++++++++++++++%7D%2C+%7B%0D%0A++++++++++++++++++++++++++++++++%22name%22%3A+%22name%22%2C%0D%0A++++++++++++++++++++++++++++++++%22type%22%3A+%22string%22%0D%0A++++++++++++++++++++++++++++%7D%2C%7B%0D%0A%09%09%09%09%09%09%09%09%22name%22%3A+%22update_time%22%2C%0D%0A%09%09%09%09%09%09%09%09%22type%22%3A+%22datetime%22%0D%0A%09%09%09%09%09%09%09%7D%0D%0A++++++++++++++++++++++++%5D%2C%0D%0A++++++++++++++++++++++++%22connection%22%3A+%5B%7B%0D%0A++++++++++++++++++++++++++++++++%22jdbcUrl%22%3A+%22jdbc%3Amysql%3A%2F%2F192.168.21.71%3A3306%2Ftest_sink%3FuseSSL%3Dfalse%22%2C%0D%0A++++++++++++++++++++++++++++++++%22table%22%3A+%5B%22test_sink1%22%5D%0D%0A++++++++++++++++++++++++++++%7D%0D%0A++++++++++++++++++++++++%5D%2C%0D%0A++++++++++++++++++++++++%22flushIntervalMills%22%3A+%223000%22%2C%0D%0A++++++++++++++++++++++++%22password%22%3A+%22root%22%2C%0D%0A++++++++++++++++++++++++%22username%22%3A+%22root%22%2C%0D%0A++++++++++++++++++++++++%22writeMode%22%3A+%22replace%22%0D%0A++++++++++++++++++++%7D%0D%0A++++++++++++++++%7D%0D%0A++++++++++++%7D%0D%0A++++++++%5D%2C%0D%0A++++++++%22setting%22%3A+%7B%0D%0A++++++++++++%22restore%22%3A+%7B%0D%0A++++++++++++++++%22restoreColumnName%22%3A+%22id%22%0D%0A++++++++++++%7D%2C%0D%0A++++++++++++%22speed%22%3A+%7B%0D%0A++++++++++++++++%22bytes%22%3A+0%2C%0D%0A++++++++++++++++%22channel%22%3A+1%0D%0A++++++++++++%7D%0D%0A++++++++%7D%0D%0A++++%7D%0D%0A%7D%0D%0A 14:32:45.670 [main] INFO com.dtstack.chunjun.Main - -jobName 14:32:45.671 [main] INFO com.dtstack.chunjun.Main - Flink_Job 14:32:45.671 [main] INFO com.dtstack.chunjun.Main - -chunjunDistDir 14:32:45.671 [main] INFO com.dtstack.chunjun.Main - F:\IDEA_Code\szhn\demo\chunjun\zip\chunjun-dist-1.12-SNAPSHOT\chunjun-dist 14:32:45.671 [main] INFO com.dtstack.chunjun.Main - -jobType 14:32:45.671 [main] INFO com.dtstack.chunjun.Main - sync 14:32:45.671 [main] INFO com.dtstack.chunjun.Main - -confProp 14:32:45.671 [main] INFO com.dtstack.chunjun.Main - {} 14:32:45.671 [main] INFO com.dtstack.chunjun.Main - -pluginLoadMode 14:32:45.671 [main] INFO com.dtstack.chunjun.Main - shipfile 14:32:45.671 [main] INFO com.dtstack.chunjun.Main - -mode 14:32:45.672 [main] INFO com.dtstack.chunjun.Main - standalone 14:32:45.672 [main] INFO com.dtstack.chunjun.Main - -flinkConfDir 14:32:45.672 [main] INFO com.dtstack.chunjun.Main - F:\IDEA_Code\szhn\demo\chunjun\zip\flink-1.12.7\conf 14:32:45.672 [main] INFO com.dtstack.chunjun.Main - ------------------------------------------- 14:32:46.024 [main] INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: jobmanager.rpc.address, localhost 14:32:46.024 [main] INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: jobmanager.rpc.port, 6123 14:32:46.024 [main] INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: jobmanager.memory.process.size, 1600m 14:32:46.025 [main] INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: taskmanager.memory.process.size, 1728m 14:32:46.025 [main] INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: taskmanager.numberOfTaskSlots, 1 14:32:46.025 [main] INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: parallelism.default, 1 14:32:46.027 [main] INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: jobmanager.execution.failover-strategy, region 14:32:46.029 [main] INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: taskmanager.cpu.cores, 2 14:32:46.030 [main] INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: taskmanager.memory.task.heap.size, 512m 14:32:46.030 [main] INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: taskmanager.memory.managed.size, 512m 14:32:46.030 [main] INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: taskmanager.memory.network.min, 64m 14:32:46.031 [main] INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: taskmanager.memory.network.max, 64m Exception in thread "main" org.apache.flink.client.program.ProgramInvocationException: The program caused an error:

Classpath: [file:/F:/IDEA_Code/szhn/demo/chunjun/zip/chunjun-dist-1.12-SNAPSHOT/chunjun-dist/chunjun-core.jar]

System.out: (none)

System.err: (none) at org.apache.flink.client.program.PackagedProgramUtils.generateException(PackagedProgramUtils.java:264) at org.apache.flink.client.program.PackagedProgramUtils.getPipelineFromProgram(PackagedProgramUtils.java:172) at org.apache.flink.client.program.PackagedProgramUtils.createJobGraph(PackagedProgramUtils.java:82) at org.apache.flink.client.program.PackagedProgramUtils.createJobGraph(PackagedProgramUtils.java:117) at com.lizhaoxia.flink_demo.FlinkDemoApplicationTests.buildJobGraph(FlinkDemoApplicationTests.java:91) at com.lizhaoxia.flink_demo.FlinkDemoApplicationTests.main(FlinkDemoApplicationTests.java:60) Caused by: java.lang.NoClassDefFoundError: org/apache/flink/table/api/bridge/java/StreamTableEnvironment at com.dtstack.chunjun.environment.EnvFactory.createStreamTableEnvironment(EnvFactory.java:69) at com.dtstack.chunjun.Main.main(Main.java:108) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:349) at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:219) at org.apache.flink.client.program.PackagedProgramUtils.getPipelineFromProgram(PackagedProgramUtils.java:158) ... 4 more Caused by: java.lang.ClassNotFoundException: org.apache.flink.table.api.bridge.java.StreamTableEnvironment at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at org.apache.flink.util.FlinkUserCodeClassLoader.loadClassWithoutExceptionHandling(FlinkUserCodeClassLoader.java:64) at org.apache.flink.util.FlinkUserCodeClassLoader.loadClass(FlinkUserCodeClassLoader.java:48) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ... 13 more Disconnected from the target VM, address: '127.0.0.1:59771', transport: 'socket'

Process finished with exit code 1 `

Code of Conduct

zoudaokoulife commented 1 year ago

ChunJun 后面近期会规划提供服务功能；包括提供restful 接口：任务的提交，停止，状态查询等； ；如果有兴趣可以一起来参与讨论开发

382035656 commented 1 year ago

ChunJun 后面近期会规划提供服务功能；包括提供restful 接口：任务的提交，停止，状态查询等； ；如果有兴趣可以一起来参与讨论开发

在哪里能参与呢？

382035656 commented 1 year ago

ChunJun 后面近期会规划提供服务功能；包括提供restful 接口：任务的提交，停止，状态查询等； ；如果有兴趣可以一起来参与讨论开发

我看了一下项目源码在Standalone模式下也是通过ClusterClient去提交的任务那理论上来说通过RestClusterClient一样能够成功提交任务吧，且我现阶段已经成功吧任务提交到flink且返回jobID但现在flink仍然报ClassNotFoundException: ch.qos.logback.classic.Logger

zoudaokoulife commented 1 year ago

ch.qos.logback.classic.Logger 这个是对你flink lib 添加下logback 的包