search

awslabs / amazon-kinesis-connector-flink

This is a fork of the Apache Flink Kinesis connector adding Enhanced Fanout support for Flink 1.8/1.11 on KDA.
Apache License 2.0
20 stars 11 forks source link

Error occurred on EFO subscription: java.nio.channels.ClosedChannelException - (null). #30

Closed ensc-yang closed 3 years ago

ensc-yang commented 3 years ago

Hello there,

Im running KDA v1.11 application with the EFO connector library version and I am experiencing an issue where the connection to the shard is closed and is unable to continue consuming from the shard. Is there any way to resolve this?


        <dependency>
            <groupId>software.amazon.kinesis</groupId>
            <artifactId>amazon-kinesis-connector-flink</artifactId>
            <version>2.0.0</version>
        </dependency>
Field Value
@ingestionTime 1621409226178
@log flink
@logStream log stream
@message {"locationInformation":"software.amazon.kinesis.connectors.flink.internals.publisher.fanout.FanOutShardSubscriber.handleError(FanOutShardSubscriber.java:215)","logger":"software.amazon.kinesis.connectors.flink.internals.publisher.fanout.FanOutShardSubscriber","message":"Error occurred on EFO subscription: java.nio.channels.ClosedChannelException - (null). shardId-000000000098 (arn:aws:kinesis:us-east-1::redacted)","throwableInformation":"java.nio.channels.ClosedChannelException\n\tat software.amazon.kinesis.shaded.io.netty.channel.AbstractChannel$AbstractUnsafe.newClosedChannelException(AbstractChannel.java:957) ~[blob_p-797a6cd4af3a7eca5d4b52d115d08a26a88fdc7b-be382e87f20893753401bbffc88b858c:?]\n\tat software.amazon.kinesis.shaded.io.netty.channel.AbstractChannel$AbstractUnsafe.write(AbstractChannel.java:865) ~[blob_p-797a6cd4af3a7eca5d4b52d115d08a26a88fdc7b-be382e87f20893753401bbffc88b858c:?]\n\tat software.amazon.kinesis.shaded.io.netty.channel.DefaultChannelPipeline$HeadContext.write(DefaultChannelPipeline.java:1367) ~[blob_p-797a6cd4af3a7eca5d4b52d115d08a26a88fdc7b-be382e87f20893753401bbffc88b858c:?]\n\tat software.amazon.kinesis.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeWrite0(AbstractChannelHandlerContext.java:715) ~[blob_p-797a6cd4af3a7eca5d4b52d115d08a26a88fdc7b-be382e87f20893753401bbffc88b858c:?]\n\tat software.amazon.kinesis.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeWrite(AbstractChannelHandlerContext.java:707) ~[blob_p-797a6cd4af3a7eca5d4b52d115d08a26a88fdc7b-be382e87f20893753401bbffc88b858c:?]\n\tat software.amazon.kinesis.shaded.io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:790) ~[blob_p-797a6cd4af3a7eca5d4b52d115d08a26a88fdc7b-be382e87f20893753401bbffc88b858c:?]\n\tat software.amazon.kinesis.shaded.io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:700) ~[blob_p-797a6cd4af3a7eca5d4b52d115d08a26a88fdc7b-be382e87f20893753401bbffc88b858c:?]\n\tat software.amazon.kinesis.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeWrite(AbstractChannelHandlerContext.java:709) ~[blob_p-797a6cd4af3a7eca5d4b52d115d08a26a88fdc7b-be382e87f20893753401bbffc88b858c:?]\n\tat software.amazon.kinesis.shaded.io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:790) ~[blob_p-797a6cd4af3a7eca5d4b52d115d08a26a88fdc7b-be382e87f20893753401bbffc88b858c:?]\n\tat software.amazon.kinesis.shaded.io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:700) ~[blob_p-797a6cd4af3a7eca5d4b52d115d08a26a88fdc7b-be382e87f20893753401bbffc88b858c:?]\n\tat software.amazon.kinesis.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeWrite(AbstractChannelHandlerContext.java:709) ~[blob_p-797a6cd4af3a7eca5d4b52d115d08a26a88fdc7b-be382e87f20893753401bbffc88b858c:?]\n\tat software.amazon.kinesis.shaded.io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:790) ~[blob_p-797a6cd4af3a7eca5d4b52d115d08a26a88fdc7b-be382e87f20893753401bbffc88b858c:?]\n\tat software.amazon.kinesis.shaded.io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:700) ~[blob_p-797a6cd4af3a7eca5d4b52d115d08a26a88fdc7b-be382e87f20893753401bbffc88b858c:?]\n\tat software.amazon.kinesis.shaded.io.netty.handler.codec.http2.AbstractHttp2StreamChannel.write0(AbstractHttp2StreamChannel.java:1093) ~[blob_p-797a6cd4af3a7eca5d4b52d115d08a26a88fdc7b-be382e87f20893753401bbffc88b858c:?]\n\tat software.amazon.kinesis.shaded.io.netty.handler.codec.http2.AbstractHttp2StreamChannel$Http2ChannelUnsafe.updateLocalWindowIfNeeded(AbstractHttp2StreamChannel.java:843) ~[blob_p-797a6cd4af3a7eca5d4b52d115d08a26a88fdc7b-be382e87f20893753401bbffc88b858c:?]\n\tat software.amazon.kinesis.shaded.io.netty.handler.codec.http2.AbstractHttp2StreamChannel$Http2ChannelUnsafe.beginRead(AbstractHttp2StreamChannel.java:783) ~[blob_p-797a6cd4af3a7eca5d4b52d115d08a26a88fdc7b-be382e87f20893753401bbffc88b858c:?]\n\tat software.amazon.kinesis.shaded.io.netty.channel.DefaultChannelPipeline$HeadContext.read(DefaultChannelPipeline.java:1362) ~[blob_p-797a6cd4af3a7eca5d4b52d115d08a26a88fdc7b-be382e87f20893753401bbffc88b858c:?]\n\tat software.amazon.kinesis.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeRead(AbstractChannelHandlerContext.java:684) ~[blob_p-797a6cd4af3a7eca5d4b52d115d08a26a88fdc7b-be382e87f20893753401bbffc88b858c:?]\n\tat software.amazon.kinesis.shaded.io.netty.channel.AbstractChannelHandlerContext.read(AbstractChannelHandlerContext.java:669) ~[blob_p-797a6cd4af3a7eca5d4b52d115d08a26a88fdc7b-be382e87f20893753401bbffc88b858c:?]\n\tat software.amazon.kinesis.shaded.software.amazon.awssdk.http.nio.netty.internal.http2.FlushOnReadHandler.read(FlushOnReadHandler.java:40) ~[blob_p-797a6cd4af3a7eca5d4b52d115d08a26a88fdc7b-be382e87f20893753401bbffc88b858c:?]\n\tat software.amazon.kinesis.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeRead(AbstractChannelHandlerContext.java:684) ~[blob_p-797a6cd4af3a7eca5d4b52d115d08a26a88fdc7b-be382e87f20893753401bbffc88b858c:?]\n\tat software.amazon.kinesis.shaded.io.netty.channel.AbstractChannelHandlerContext.read(AbstractChannelHandlerContext.java:669) ~[blob_p-797a6cd4af3a7eca5d4b52d115d08a26a88fdc7b-be382e87f20893753401bbffc88b858c:?]\n\tat software.amazon.kinesis.shaded.software.amazon.awssdk.http.nio.netty.internal.nrs.HandlerPublisher.requestDemand(HandlerPublisher.java:130) ~[blob_p-797a6cd4af3a7eca5d4b52d115d08a26a88fdc7b-be382e87f20893753401bbffc88b858c:?]\n\tat software.amazon.kinesis.shaded.software.amazon.awssdk.http.nio.netty.internal.nrs.HttpStreamsHandler$1.requestDemand(HttpStreamsHandler.java:191) ~[blob_p-797a6cd4af3a7eca5d4b52d115d08a26a88fdc7b-be382e87f20893753401bbffc88b858c:?]\n\tat software.amazon.kinesis.shaded.software.amazon.awssdk.http.nio.netty.internal.nrs.HandlerPublisher.receivedDemand(HandlerPublisher.java:304) ~[blob_p-797a6cd4af3a7eca5d4b52d115d08a26a88fdc7b-be382e87f20893753401bbffc88b858c:?]\n\tat software.amazon.kinesis.shaded.software.amazon.awssdk.http.nio.netty.internal.nrs.HandlerPublisher.access$200(HandlerPublisher.java:61) ~[blob_p-797a6cd4af3a7eca5d4b52d115d08a26a88fdc7b-be382e87f20893753401bbffc88b858c:?]\n\tat software.amazon.kinesis.shaded.software.amazon.awssdk.http.nio.netty.internal.nrs.HandlerPublisher$ChannelSubscription$1.run(HandlerPublisher.java:495) ~[blob_p-797a6cd4af3a7eca5d4b52d115d08a26a88fdc7b-be382e87f20893753401bbffc88b858c:?]\n\tat software.amazon.kinesis.shaded.io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164) ~[blob_p-797a6cd4af3a7eca5d4b52d115d08a26a88fdc7b-be382e87f20893753401bbffc88b858c:?]\n\tat software.amazon.kinesis.shaded.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472) ~[blob_p-797a6cd4af3a7eca5d4b52d115d08a26a88fdc7b-be382e87f20893753401bbffc88b858c:?]\n\tat software.amazon.kinesis.shaded.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500) ~[blob_p-797a6cd4af3a7eca5d4b52d115d08a26a88fdc7b-be382e87f20893753401bbffc88b858c:?]\n\tat software.amazon.kinesis.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[blob_p-797a6cd4af3a7eca5d4b52d115d08a26a88fdc7b-be382e87f20893753401bbffc88b858c:?]\n\tat software.amazon.kinesis.shaded.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[blob_p-797a6cd4af3a7eca5d4b52d115d08a26a88fdc7b-be382e87f20893753401bbffc88b858c:?]\n\tat java.lang.Thread.run(Thread.java:829) [?:?]\n","threadName":"shardConsumers-Source: Custom Source -> kinesis-source -> time-range-filter (1/2)-thread-7","applicationARN":"arn:aws:kinesisanalytics:us-east-1::redacted","applicationVersionId":"86","messageSchemaVersion":"1","messageType":"WARN"}
@timestamp 1621409222000
applicationARN arn:aws:kinesisanalytics:us-east-1::redacted
applicationVersionId 86
locationInformation software.amazon.kinesis.connectors.flink.internals.publisher.fanout.FanOutShardSubscriber.handleError(FanOutShardSubscriber.java:215)
logger software.amazon.kinesis.connectors.flink.internals.publisher.fanout.FanOutShardSubscriber
message Error occurred on EFO subscription: java.nio.channels.ClosedChannelException - (null). shardId-000000000098 (arn:aws:kinesis:us-east-1::redacted)
messageSchemaVersion 1
messageType WARN
threadName shardConsumers-Source: Custom Source -> kinesis-source -> time-range-filter (1/2)-thread-7
throwableInformation java.nio.channels.ClosedChannelExceptionat software.amazon.kinesis.shaded.io.netty.channel.AbstractChannel$AbstractUnsafe.newClosedChannelException(AbstractChannel.java:957) ~[blob_p-797a6cd4af3a7eca5d4b52d115d08a26a88fdc7b-be382e87f20893753401bbffc88b858c:?]at software.amazon.kinesis.shaded.io.netty.channel.AbstractChannel$AbstractUnsafe.write(AbstractChannel.java:865) ~[blob_p-797a6cd4af3a7eca5d4b52d115d08a26a88fdc7b-be382e87f20893753401bbffc88b858c:?]at software.amazon.kinesis.shaded.io.netty.channel.DefaultChannelPipeline$HeadContext.write(DefaultChannelPipeline.java:1367) ~[blob_p-797a6cd4af3a7eca5d4b52d115d08a26a88fdc7b-be382e87f20893753401bbffc88b858c:?]at software.amazon.kinesis.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeWrite0(AbstractChannelHandlerContext.java:715) ~[blob_p-797a6cd4af3a7eca5d4b52d115d08a26a88fdc7b-be382e87f20893753401bbffc88b858c:?]at software.amazon.kinesis.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeWrite(AbstractChannelHandlerContext.java:707) ~[blob_p-797a6cd4af3a7eca5d4b52d115d08a26a88fdc7b-be382e87f20893753401bbffc88b858c:?]at software.amazon.kinesis.shaded.io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:790) ~[blob_p-797a6cd4af3a7eca5d4b52d115d08a26a88fdc7b-be382e87f20893753401bbffc88b858c:?]at software.amazon.kinesis.shaded.io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:700) ~[blob_p-797a6cd4af3a7eca5d4b52d115d08a26a88fdc7b-be382e87f20893753401bbffc88b858c:?]at software.amazon.kinesis.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeWrite(AbstractChannelHandlerContext.java:709) ~[blob_p-797a6cd4af3a7eca5d4b52d115d08a26a88fdc7b-be382e87f20893753401bbffc88b858c:?]at software.amazon.kinesis.shaded.io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:790) ~[blob_p-797a6cd4af3a7eca5d4b52d115d08a26a88fdc7b-be382e87f20893753401bbffc88b858c:?]at software.amazon.kinesis.shaded.io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:700) ~[blob_p-797a6cd4af3a7eca5d4b52d115d08a26a88fdc7b-be382e87f20893753401bbffc88b858c:?]at software.amazon.kinesis.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeWrite(AbstractChannelHandlerContext.java:709) ~[blob_p-797a6cd4af3a7eca5d4b52d115d08a26a88fdc7b-be382e87f20893753401bbffc88b858c:?]at software.amazon.kinesis.shaded.io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:790) ~[blob_p-797a6cd4af3a7eca5d4b52d115d08a26a88fdc7b-be382e87f20893753401bbffc88b858c:?]at software.amazon.kinesis.shaded.io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:700) ~[blob_p-797a6cd4af3a7eca5d4b52d115d08a26a88fdc7b-be382e87f20893753401bbffc88b858c:?]at software.amazon.kinesis.shaded.io.netty.handler.codec.http2.AbstractHttp2StreamChannel.write0(AbstractHttp2StreamChannel.java:1093) ~[blob_p-797a6cd4af3a7eca5d4b52d115d08a26a88fdc7b-be382e87f20893753401bbffc88b858c:?]at software.amazon.kinesis.shaded.io.netty.handler.codec.http2.AbstractHttp2StreamChannel$Http2ChannelUnsafe.updateLocalWindowIfNeeded(AbstractHttp2StreamChannel.java:843) ~[blob_p-797a6cd4af3a7eca5d4b52d115d08a26a88fdc7b-be382e87f20893753401bbffc88b858c:?]at software.amazon.kinesis.shaded.io.netty.handler.codec.http2.AbstractHttp2StreamChannel$Http2ChannelUnsafe.beginRead(AbstractHttp2StreamChannel.java:783) ~[blob_p-797a6cd4af3a7eca5d4b52d115d08a26a88fdc7b-be382e87f20893753401bbffc88b858c:?]at software.amazon.kinesis.shaded.io.netty.channel.DefaultChannelPipeline$HeadContext.read(DefaultChannelPipeline.java:1362) ~[blob_p-797a6cd4af3a7eca5d4b52d115d08a26a88fdc7b-be382e87f20893753401bbffc88b858c:?]at software.amazon.kinesis.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeRead(AbstractChannelHandlerContext.java:684) ~[blob_p-797a6cd4af3a7eca5d4b52d115d08a26a88fdc7b-be382e87f20893753401bbffc88b858c:?]at software.amazon.kinesis.shaded.io.netty.channel.AbstractChannelHandlerContext.read(AbstractChannelHandlerContext.java:669) ~[blob_p-797a6cd4af3a7eca5d4b52d115d08a26a88fdc7b-be382e87f20893753401bbffc88b858c:?]at software.amazon.kinesis.shaded.software.amazon.awssdk.http.nio.netty.internal.http2.FlushOnReadHandler.read(FlushOnReadHandler.java:40) ~[blob_p-797a6cd4af3a7eca5d4b52d115d08a26a88fdc7b-be382e87f20893753401bbffc88b858c:?]at software.amazon.kinesis.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeRead(AbstractChannelHandlerContext.java:684) ~[blob_p-797a6cd4af3a7eca5d4b52d115d08a26a88fdc7b-be382e87f20893753401bbffc88b858c:?]at software.amazon.kinesis.shaded.io.netty.channel.AbstractChannelHandlerContext.read(AbstractChannelHandlerContext.java:669) ~[blob_p-797a6cd4af3a7eca5d4b52d115d08a26a88fdc7b-be382e87f20893753401bbffc88b858c:?]at software.amazon.kinesis.shaded.software.amazon.awssdk.http.nio.netty.internal.nrs.HandlerPublisher.requestDemand(HandlerPublisher.java:130) ~[blob_p-797a6cd4af3a7eca5d4b52d115d08a26a88fdc7b-be382e87f20893753401bbffc88b858c:?]at software.amazon.kinesis.shaded.software.amazon.awssdk.http.nio.netty.internal.nrs.HttpStreamsHandler$1.requestDemand(HttpStreamsHandler.java:191) ~[blob_p-797a6cd4af3a7eca5d4b52d115d08a26a88fdc7b-be382e87f20893753401bbffc88b858c:?]at software.amazon.kinesis.shaded.software.amazon.awssdk.http.nio.netty.internal.nrs.HandlerPublisher.receivedDemand(HandlerPublisher.java:304) ~[blob_p-797a6cd4af3a7eca5d4b52d115d08a26a88fdc7b-be382e87f20893753401bbffc88b858c:?]at software.amazon.kinesis.shaded.software.amazon.awssdk.http.nio.netty.internal.nrs.HandlerPublisher.access$200(HandlerPublisher.java:61) ~[blob_p-797a6cd4af3a7eca5d4b52d115d08a26a88fdc7b-be382e87f20893753401bbffc88b858c:?]at software.amazon.kinesis.shaded.software.amazon.awssdk.http.nio.netty.internal.nrs.HandlerPublisher$ChannelSubscription$1.run(HandlerPublisher.java:495) ~[blob_p-797a6cd4af3a7eca5d4b52d115d08a26a88fdc7b-be382e87f20893753401bbffc88b858c:?]at software.amazon.kinesis.shaded.io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164) ~[blob_p-797a6cd4af3a7eca5d4b52d115d08a26a88fdc7b-be382e87f20893753401bbffc88b858c:?]at software.amazon.kinesis.shaded.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472) ~[blob_p-797a6cd4af3a7eca5d4b52d115d08a26a88fdc7b-be382e87f20893753401bbffc88b858c:?]at software.amazon.kinesis.shaded.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500) ~[blob_p-797a6cd4af3a7eca5d4b52d115d08a26a88fdc7b-be382e87f20893753401bbffc88b858c:?]at software.amazon.kinesis.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[blob_p-797a6cd4af3a7eca5d4b52d115d08a26a88fdc7b-be382e87f20893753401bbffc88b858c:?]at software.amazon.kinesis.shaded.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[blob_p-797a6cd4af3a7eca5d4b52d115d08a26a88fdc7b-be382e87f20893753401bbffc88b858c:?]at java.lang.Thread.run(Thread.java:829) [?:?]
dannycranmer commented 3 years ago

@ensc-yang I believe this error has been fixed in v2.0.1. Can you please try upgrading to v2.0.3 and let us know if the problem persists?

dannycranmer commented 3 years ago

Closing due to staleness, please reopen if needed