connection refused by submited flink cluster to docker Hadoop cluster on macOS.

I'm using docker-compose.yml bootstrap Hadoop cluster on macOS as my local develop environment and try to submit flink job to it. This is my docker-compose.yml and env: https://github.com/flowerfine/scaleph/tree/dev/tools/docker/hadoop

To run successfully on macOS, I had added ${host ip} datanode namenode ... ${namenode container id} ${datanode container id} ... to my local hosts file, then I can browse HDFS from web-ui, submit flink application or per-job deploy mode job, create session cluster. But when I try to access submited flink cluster, such as list jobs or submit flink job to created session cluster, exception encontered:

Caused by: org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: 7c8c951d151d/127.0.0.1:41565
Caused by: java.net.ConnectException: Connection refused
    at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779)
    at org.apache.flink.shaded.netty4.io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:330)
    at org.apache.flink.shaded.netty4.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:334)
    at org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:707)
    at org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:655)
    at org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:581)
    at org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
    at org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
    at org.apache.flink.shaded.netty4.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
    at java.base/java.lang.Thread.run(Thread.java:834)

Notice that every attemp will fail on another port, not always 127.0.0.1:41565. Can any one give me some advice? I will appreciate your help

big-data-europe / docker-hadoop

connection refused by submited flink cluster to docker Hadoop cluster on macOS. #131