search

flink-extended / flink-remote-shuffle

Remote Shuffle Service for Flink
Apache License 2.0
191 stars 57 forks source link

Conflict with Flink 1.15.4 #94

Open dangshazi opened 1 year ago

dangshazi commented 1 year ago

Flink changes ConnectionID in 1.15.4. Related issue : #21358

Error msg:

java.lang.NoSuchMethodError: org.apache.flink.runtime.io.network.ConnectionID.<init>(Ljava/net/InetSocketAddress;I)V
    at com.alibaba.flink.shuffle.plugin.transfer.RemoteShuffleInputGate$FakedRemoteInputChannel.<init>(RemoteShuffleInputGate.java:726)
    at com.alibaba.flink.shuffle.plugin.transfer.RemoteShuffleInputGate.getChannel(RemoteShuffleInputGate.java:682)
    at org.apache.flink.runtime.io.network.partition.consumer.InputGate.setChannelStateWriter(InputGate.java:90)
    at org.apache.flink.runtime.taskmanager.InputGateWithMetrics.setChannelStateWriter(InputGateWithMetrics.java:120)
    at org.apache.flink.streaming.runtime.tasks.StreamTask.injectChannelStateWriterIntoChannels(StreamTask.java:491)
    at org.apache.flink.streaming.runtime.tasks.StreamTask.<init>(StreamTask.java:464)
    at org.apache.flink.streaming.runtime.tasks.StreamTask.<init>(StreamTask.java:363)
    at org.apache.flink.streaming.runtime.tasks.StreamTask.<init>(StreamTask.java:336)
    at org.apache.flink.streaming.runtime.tasks.StreamTask.<init>(StreamTask.java:328)
    at org.apache.flink.streaming.runtime.tasks.StreamTask.<init>(StreamTask.java:318)
    at org.apache.flink.streaming.runtime.tasks.AbstractTwoInputStreamTask.<init>(AbstractTwoInputStreamTask.java:50)
    at org.apache.flink.streaming.runtime.tasks.TwoInputStreamTask.<init>(TwoInputStreamTask.java:49)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
    at org.apache.flink.runtime.taskmanager.Task.loadAndInstantiateInvokable(Task.java:1612)
    at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:737)
    at org.apache.flink.runtime.taskmanager.Task.run(Task.java:569)
    at java.lang.Thread.run(Thread.java:748)
TanYuxin-tyx commented 1 year ago

Yes that's right.

If you want to compile with new Flink 1.15, you can use the patch in the main branch. https://github.com/flink-extended/flink-remote-shuffle/commit/23f3fbfa2bfe264d86d1afe3aaceb1c3fd48e36a

dangshazi commented 1 year ago

Thanks