search

DataLinkDC / dinky

Dinky is a real-time data development platform based on Apache Flink, enabling agile data development, deployment and operation.
http://www.dinky.org.cn
Apache License 2.0
3.14k stars 1.16k forks source link

java.lang.ArrayStoreException occur in yarn application mode #2342

Closed hushenmin closed 1 year ago

hushenmin commented 1 year ago

Search before asking

What happened

2023-09-26 11:07:13,752 WARN org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap [] - Application failed unexpectedly: java.util.concurrent.CompletionException: org.apache.flink.client.deployment.application.ApplicationExecutionException: Could not execute application. at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:292) ~[?:1.8.0_221] at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:308) ~[?:1.8.0_221] at java.util.concurrent.CompletableFuture.uniCompose(CompletableFuture.java:943) ~[?:1.8.0_221] at java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:926) ~[?:1.8.0_221] at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474) ~[?:1.8.0_221] at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1977) ~[?:1.8.0_221] at org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap.runApplicationEntryPoint(ApplicationDispatcherBootstrap.java:337) ~[flink-dist-1.16.1.jar:1.16.1] at org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap.lambda$runApplicationAsync$2(ApplicationDispatcherBootstrap.java:254) ~[flink-dist-1.16.1.jar:1.16.1] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_221] at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_221] at org.apache.flink.runtime.concurrent.akka.ActorSystemScheduledExecutorAdapter$ScheduledFutureTask.run(ActorSystemScheduledExecutorAdapter.java:171) ~[flink-rpc-akka_5f6e0abe-8e83-40a8-9289-19705d8ad120.jar:1.16.1] at org.apache.flink.runtime.concurrent.akka.ClassLoadingUtils.runWithContextClassLoader(ClassLoadingUtils.java:68) ~[flink-rpc-akka_5f6e0abe-8e83-40a8-9289-19705d8ad120.jar:1.16.1] at org.apache.flink.runtime.concurrent.akka.ClassLoadingUtils.lambda$withContextClassLoader$0(ClassLoadingUtils.java:41) ~[flink-rpc-akka_5f6e0abe-8e83-40a8-9289-19705d8ad120.jar:1.16.1] at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:49) [flink-rpc-akka_5f6e0abe-8e83-40a8-9289-19705d8ad120.jar:1.16.1] at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:48) [flink-rpc-akka_5f6e0abe-8e83-40a8-9289-19705d8ad120.jar:1.16.1] at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289) [?:1.8.0_221] at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056) [?:1.8.0_221] at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692) [?:1.8.0_221] at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157) [?:1.8.0_221] Caused by: org.apache.flink.client.deployment.application.ApplicationExecutionException: Could not execute application. ... 13 more Caused by: java.lang.ExceptionInInitializerError at com.dlink.app.flinksql.Submiter.submit(Submiter.java:182) ~[dinky-app.jar:?] at com.dlink.app.MainApp.main(MainApp.java:44) ~[dinky-app.jar:?] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_221] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_221] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_221] at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_221] at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:355) ~[flink-dist-1.16.1.jar:1.16.1] at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:222) ~[flink-dist-1.16.1.jar:1.16.1] at org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:98) ~[flink-dist-1.16.1.jar:1.16.1] at org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap.runApplicationEntryPoint(ApplicationDispatcherBootstrap.java:301) ~[flink-dist-1.16.1.jar:1.16.1] ... 12 more Caused by: java.lang.ArrayStoreException at java.lang.System.arraycopy(Native Method) ~[?:1.8.0_221] at java.util.stream.SpinedBuffer.copyInto(SpinedBuffer.java:194) ~[?:1.8.0_221] at java.util.stream.Nodes$SpinedNodeBuilder.copyInto(Nodes.java:1290) ~[?:1.8.0_221] at java.util.stream.SpinedBuffer.asArray(SpinedBuffer.java:215) ~[?:1.8.0_221] at java.util.stream.Nodes$SpinedNodeBuilder.asArray(Nodes.java:1296) ~[?:1.8.0_221] at java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:439) ~[?:1.8.0_221] at com.dlink.trans.Operations.getAllOperations(Operations.java:58) ~[dinky-app.jar:?] at com.dlink.trans.Operations.(Operations.java:45) ~[dinky-app.jar:?] at com.dlink.app.flinksql.Submiter.submit(Submiter.java:182) ~[dinky-app.jar:?] at com.dlink.app.MainApp.main(MainApp.java:44) ~[dinky-app.jar:?] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_221] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_221] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_221] at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_221] at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:355) ~[flink-dist-1.16.1.jar:1.16.1] at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:222) ~[flink-dist-1.16.1.jar:1.16.1] at org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:98) ~[flink-dist-1.16.1.jar:1.16.1] at org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap.runApplicationEntryPoint(ApplicationDispatcherBootstrap.java:301) ~[flink-dist-1.16.1.jar:1.16.1] ... 12 more 2023-09-26 11:07:13,761 ERROR org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Fatal error occurred in the cluster entrypoint. java.util.concurrent.CompletionException: org.apache.flink.client.deployment.application.ApplicationExecutionException: Could not execute application. at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:292) ~[?:1.8.0_221] at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:308) ~[?:1.8.0_221] at java.util.concurrent.CompletableFuture.uniCompose(CompletableFuture.java:943) ~[?:1.8.0_221] at java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:926) ~[?:1.8.0_221] at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474) ~[?:1.8.0_221] at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1977) ~[?:1.8.0_221] at org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap.runApplicationEntryPoint(ApplicationDispatcherBootstrap.java:337) ~[flink-dist-1.16.1.jar:1.16.1] at org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap.lambda$runApplicationAsync$2(ApplicationDispatcherBootstrap.java:254) ~[flink-dist-1.16.1.jar:1.16.1] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_221] at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_221] at org.apache.flink.runtime.concurrent.akka.ActorSystemScheduledExecutorAdapter$ScheduledFutureTask.run(ActorSystemScheduledExecutorAdapter.java:171) ~[flink-rpc-akka_5f6e0abe-8e83-40a8-9289-19705d8ad120.jar:1.16.1] at org.apache.flink.runtime.concurrent.akka.ClassLoadingUtils.runWithContextClassLoader(ClassLoadingUtils.java:68) ~[flink-rpc-akka_5f6e0abe-8e83-40a8-9289-19705d8ad120.jar:1.16.1] at org.apache.flink.runtime.concurrent.akka.ClassLoadingUtils.lambda$withContextClassLoader$0(ClassLoadingUtils.java:41) ~[flink-rpc-akka_5f6e0abe-8e83-40a8-9289-19705d8ad120.jar:1.16.1] at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:49) [flink-rpc-akka_5f6e0abe-8e83-40a8-9289-19705d8ad120.jar:1.16.1] at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:48) [flink-rpc-akka_5f6e0abe-8e83-40a8-9289-19705d8ad120.jar:1.16.1] at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289) [?:1.8.0_221] at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056) [?:1.8.0_221] at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692) [?:1.8.0_221] at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157) [?:1.8.0_221] Caused by: org.apache.flink.client.deployment.application.ApplicationExecutionException: Could not execute application. ... 13 more Caused by: java.lang.ExceptionInInitializerError at com.dlink.app.flinksql.Submiter.submit(Submiter.java:182) ~[dinky-app.jar:?] at com.dlink.app.MainApp.main(MainApp.java:44) ~[dinky-app.jar:?] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_221] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_221] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_221] at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_221] at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:355) ~[flink-dist-1.16.1.jar:1.16.1] at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:222) ~[flink-dist-1.16.1.jar:1.16.1] at org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:98) ~[flink-dist-1.16.1.jar:1.16.1] at org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap.runApplicationEntryPoint(ApplicationDispatcherBootstrap.java:301) ~[flink-dist-1.16.1.jar:1.16.1] ... 12 more Caused by: java.lang.ArrayStoreException at java.lang.System.arraycopy(Native Method) ~[?:1.8.0_221] at java.util.stream.SpinedBuffer.copyInto(SpinedBuffer.java:194) ~[?:1.8.0_221] at java.util.stream.Nodes$SpinedNodeBuilder.copyInto(Nodes.java:1290) ~[?:1.8.0_221] at java.util.stream.SpinedBuffer.asArray(SpinedBuffer.java:215) ~[?:1.8.0_221] at java.util.stream.Nodes$SpinedNodeBuilder.asArray(Nodes.java:1296) ~[?:1.8.0_221] at java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:439) ~[?:1.8.0_221] at com.dlink.trans.Operations.getAllOperations(Operations.java:58) ~[dinky-app.jar:?] at com.dlink.trans.Operations.(Operations.java:45) ~[dinky-app.jar:?] at com.dlink.app.flinksql.Submiter.submit(Submiter.java:182) ~[dinky-app.jar:?] at com.dlink.app.MainApp.main(MainApp.java:44) ~[dinky-app.jar:?] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_221] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_221] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_221] at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_221] at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:355) ~[flink-dist-1.16.1.jar:1.16.1] at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:222) ~[flink-dist-1.16.1.jar:1.16.1] at org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:98) ~[flink-dist-1.16.1.jar:1.16.1] at org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap.runApplicationEntryPoint(ApplicationDispatcherBootstrap.java:301) ~[flink-dist-1.16.1.jar:1.16.1] ... 12 more

What you expected to happen

If a subclass's constructor throws an exception, it may lead to an ArrayStoreException in the toArray method. This occurs because the toArray method attempts to wrap this exception as a RuntimeException, but ArrayStoreException is not a subclass of RuntimeException, which results in this exception being thrown.

private static Operation[] getAllOperations() { Reflections reflections = new Reflections(Operation.class.getPackage().getName()); Set<Class<?>> operations = reflections.get(Scanners.SubTypes.of(Operation.class).asClass()); return operations.stream().filter(t -> !t.isInterface()).map(t -> { try { return t.getConstructor().newInstance(); } catch (InstantiationException | IllegalAccessException | InvocationTargetException | NoSuchMethodException e) { log.error(String.format("getAllOperations error, class %s, err: %s", t, e)); throw new RuntimeException(e); } }).toArray(Operation[]::new); }

How to reproduce

flink on yarn application mode，Run flink sql statement

Anything else

No response

Version

0.7.3

Are you willing to submit PR?

Code of Conduct

aiwenmo commented 1 year ago

set classloader.resolve-order: parent-first