search

aws / aws-sdk-java

The official AWS SDK for Java 1.x. The AWS SDK for Java 2.x is available here: https://github.com/aws/aws-sdk-java-v2/
https://aws.amazon.com/sdkforjava
Apache License 2.0
4.11k stars 2.83k forks source link

com.amazonaws.SdkClientException: Unable to execute HTTP request: Broken pipe (Write failed) #2328

Closed vk9551 closed 4 years ago

vk9551 commented 4 years ago

Using 1.11.+ version of AWS-java-sdk for sqs client, with socket timeout as 22000mills and connection timeout as 2000mills.

While executing receive message we are passing wait time seconds as 5. After serving for day and half the Ec2 service throwing above exception.

com.amazonaws.SdkClientException: Unable to execute HTTP request: Broken pipe (Write failed) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleRetryableException(AmazonHttpClient.java:1207) ~[aws-java-sdk-core-1.11.782.jar!/:?] at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1153) ~[aws-java-sdk-core-1.11.782.jar!/:?] at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:802) ~[aws-java-sdk-core-1.11.782.jar!/:?] at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:770) ~[aws-java-sdk-core-1.11.782.jar!/:?] at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:744) ~[aws-java-sdk-core-1.11.782.jar!/:?] at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:704) ~[aws-java-sdk-core-1.11.782.jar!/:?] at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:686) ~[aws-java-sdk-core-1.11.782.jar!/:?] at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:550) ~[aws-java-sdk-core-1.11.782.jar!/:?] at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:530) ~[aws-java-sdk-core-1.11.782.jar!/:?] at com.amazonaws.services.sqs.AmazonSQSClient.doInvoke(AmazonSQSClient.java:2207) ~[aws-java-sdk-sqs-1.11.782.jar!/:?] at com.amazonaws.services.sqs.AmazonSQSClient.invoke(AmazonSQSClient.java:2174) ~[aws-java-sdk-sqs-1.11.782.jar!/:?] at com.amazonaws.services.sqs.AmazonSQSClient.invoke(AmazonSQSClient.java:2163) ~[aws-java-sdk-sqs-1.11.782.jar!/:?] at com.amazonaws.services.sqs.AmazonSQSClient.executeReceiveMessage(AmazonSQSClient.java:1607) ~[aws-java-sdk-sqs-1.11.782.jar!/:?] at com.amazonaws.services.sqs.AmazonSQSClient.receiveMessage(AmazonSQSClient.java:1578) ~[aws-java-sdk-sqs-1.11.782.jar!/:?] at com.amazon.sqs.javamessaging.AmazonSQSExtendedClientBase.receiveMessage(AmazonSQSExtendedClientBase.java:193) ~[amazon-sqs-java-extended-client-lib-1.0.0.jar!/:?] at com.amazon.sqs.javamessaging.AmazonSQSExtendedClient.receiveMessage(AmazonSQSExtendedClient.java:324) ~[amazon-sqs-java-extended-client-lib-1.0.0.jar!/:?] at com.nike.notificationstreamsqs.subscription.SQSConsumer.run(SQSConsumer.java:208) ~[notification-stream-sqs-1.0.3.7.jar!/:?] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[?:?] at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?] at java.lang.Thread.run(Thread.java:834) [?:?] Caused by: javax.net.ssl.SSLException: Broken pipe (Write failed) at sun.security.ssl.Alert.createSSLException(Alert.java:127) ~[?:?] at sun.security.ssl.TransportContext.fatal(TransportContext.java:320) ~[?:?] at sun.security.ssl.TransportContext.fatal(TransportContext.java:263) ~[?:?] at sun.security.ssl.TransportContext.fatal(TransportContext.java:258) ~[?:?] at sun.security.ssl.SSLSocketImpl$AppOutputStream.write(SSLSocketImpl.java:987) ~[?:?] at org.apache.http.impl.io.SessionOutputBufferImpl.streamWrite(SessionOutputBufferImpl.java:124) ~[httpcore-4.4.13.jar!/:4.4.13] at org.apache.http.impl.io.SessionOutputBufferImpl.write(SessionOutputBufferImpl.java:160) ~[httpcore-4.4.13.jar!/:4.4.13] at org.apache.http.impl.io.ContentLengthOutputStream.write(ContentLengthOutputStream.java:113) ~[httpcore-4.4.13.jar!/:4.4.13] at org.apache.http.impl.io.ContentLengthOutputStream.write(ContentLengthOutputStream.java:120) ~[httpcore-4.4.13.jar!/:4.4.13] at org.apache.http.entity.StringEntity.writeTo(StringEntity.java:167) ~[httpcore-4.4.13.jar!/:4.4.13] at org.apache.http.impl.DefaultBHttpClientConnection.sendRequestEntity(DefaultBHttpClientConnection.java:156) ~[httpcore-4.4.13.jar!/:4.4.13] at org.apache.http.impl.conn.CPoolProxy.sendRequestEntity(CPoolProxy.java:152) ~[httpclient-4.5.10.jar!/:4.5.10] at org.apache.http.protocol.HttpRequestExecutor.doSendRequest(HttpRequestExecutor.java:238) ~[httpcore-4.4.13.jar!/:4.4.13] at com.amazonaws.http.protocol.SdkHttpRequestExecutor.doSendRequest(SdkHttpRequestExecutor.java:63) ~[aws-java-sdk-core-1.11.782.jar!/:?] at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:123) ~[httpcore-4.4.13.jar!/:4.4.13] at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) ~[httpclient-4.5.10.jar!/:4.5.10] at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) ~[httpclient-4.5.10.jar!/:4.5.10] at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) ~[httpclient-4.5.10.jar!/:4.5.10] at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) ~[httpclient-4.5.10.jar!/:4.5.10] at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56) ~[httpclient-4.5.10.jar!/:4.5.10] at com.amazonaws.http.apache.client.impl.SdkHttpClient.execute(SdkHttpClient.java:72) ~[aws-java-sdk-core-1.11.782.jar!/:?] at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1330) ~[aws-java-sdk-core-1.11.782.jar!/:?] at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1145) ~[aws-java-sdk-core-1.11.782.jar!/:?] ... 20 more Suppressed: java.net.SocketException: Broken pipe (Write failed) at java.net.SocketOutputStream.socketWrite0(Native Method) ~[?:?] at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:110) ~[?:?] at java.net.SocketOutputStream.write(SocketOutputStream.java:150) ~[?:?] at sun.security.ssl.SSLSocketOutputRecord.encodeAlert(SSLSocketOutputRecord.java:81) ~[?:?] at sun.security.ssl.TransportContext.fatal(TransportContext.java:351) ~[?:?] at sun.security.ssl.TransportContext.fatal(TransportContext.java:263) ~[?:?] at sun.security.ssl.TransportContext.fatal(TransportContext.java:258) ~[?:?] at sun.security.ssl.SSLSocketImpl$AppOutputStream.write(SSLSocketImpl.java:987) ~[?:?] at org.apache.http.impl.io.SessionOutputBufferImpl.streamWrite(SessionOutputBufferImpl.java:124) ~[httpcore-4.4.13.jar!/:4.4.13] at org.apache.http.impl.io.SessionOutputBufferImpl.write(SessionOutputBufferImpl.java:160) ~[httpcore-4.4.13.jar!/:4.4.13] at org.apache.http.impl.io.ContentLengthOutputStream.write(ContentLengthOutputStream.java:113) ~[httpcore-4.4.13.jar!/:4.4.13] at org.apache.http.impl.io.ContentLengthOutputStream.write(ContentLengthOutputStream.java:120) ~[httpcore-4.4.13.jar!/:4.4.13] at org.apache.http.entity.StringEntity.writeTo(StringEntity.java:167) ~[httpcore-4.4.13.jar!/:4.4.13] at org.apache.http.impl.DefaultBHttpClientConnection.sendRequestEntity(DefaultBHttpClientConnection.java:156) ~[httpcore-4.4.13.jar!/:4.4.13] at org.apache.http.impl.conn.CPoolProxy.sendRequestEntity(CPoolProxy.java:152) ~[httpclient-4.5.10.jar!/:4.5.10] at org.apache.http.protocol.HttpRequestExecutor.doSendRequest(HttpRequestExecutor.java:238) ~[httpcore-4.4.13.jar!/:4.4.13] at com.amazonaws.http.protocol.SdkHttpRequestExecutor.doSendRequest(SdkHttpRequestExecutor.java:63) ~[aws-java-sdk-core-1.11.782.jar!/:?] at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:123) ~[httpcore-4.4.13.jar!/:4.4.13] at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) ~[httpclient-4.5.10.jar!/:4.5.10] at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) ~[httpclient-4.5.10.jar!/:4.5.10] at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) ~[httpclient-4.5.10.jar!/:4.5.10] at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) ~[httpclient-4.5.10.jar!/:4.5.10] at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56) ~[httpclient-4.5.10.jar!/:4.5.10] at com.amazonaws.http.apache.client.impl.SdkHttpClient.execute(SdkHttpClient.java:72) ~[aws-java-sdk-core-1.11.782.jar!/:?] at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1330) ~[aws-java-sdk-core-1.11.782.jar!/:?] at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1145) ~[aws-java-sdk-core-1.11.782.jar!/:?] at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:802) ~[aws-java-sdk-core-1.11.782.jar!/:?] at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:770) ~[aws-java-sdk-core-1.11.782.jar!/:?] at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:744) ~[aws-java-sdk-core-1.11.782.jar!/:?] at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:704) ~[aws-java-sdk-core-1.11.782.jar!/:?] at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:686) ~[aws-java-sdk-core-1.11.782.jar!/:?] at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:550) ~[aws-java-sdk-core-1.11.782.jar!/:?] at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:530) ~[aws-java-sdk-core-1.11.782.jar!/:?] at com.amazonaws.services.sqs.AmazonSQSClient.doInvoke(AmazonSQSClient.java:2207) ~[aws-java-sdk-sqs-1.11.782.jar!/:?] at com.amazonaws.services.sqs.AmazonSQSClient.invoke(AmazonSQSClient.java:2174) ~[aws-java-sdk-sqs-1.11.782.jar!/:?] at com.amazonaws.services.sqs.AmazonSQSClient.invoke(AmazonSQSClient.java:2163) ~[aws-java-sdk-sqs-1.11.782.jar!/:?] at com.amazonaws.services.sqs.AmazonSQSClient.executeReceiveMessage(AmazonSQSClient.java:1607) ~[aws-java-sdk-sqs-1.11.782.jar!/:?] at com.amazonaws.services.sqs.AmazonSQSClient.receiveMessage(AmazonSQSClient.java:1578) ~[aws-java-sdk-sqs-1.11.782.jar!/:?] at com.amazon.sqs.javamessaging.AmazonSQSExtendedClientBase.receiveMessage(AmazonSQSExtendedClientBase.java:193) ~[amazon-sqs-java-extended-client-lib-1.0.0.jar!/:?] at com.amazon.sqs.javamessaging.AmazonSQSExtendedClient.receiveMessage(AmazonSQSExtendedClient.java:324) ~[amazon-sqs-java-extended-client-lib-1.0.0.jar!/:?] at com.nike.notificationstreamsqs.subscription.SQSConsumer.run(SQSConsumer.java:208) ~[notification-stream-sqs-1.0.3.7.jar!/:?] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[?:?] at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?] at java.lang.Thread.run(Thread.java:834) [?:?] Caused by: java.net.SocketException: Broken pipe (Write failed) at java.net.SocketOutputStream.socketWrite0(Native Method) ~[?:?] at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:110) ~[?:?] at java.net.SocketOutputStream.write(SocketOutputStream.java:150) ~[?:?] at sun.security.ssl.SSLSocketOutputRecord.deliver(SSLSocketOutputRecord.java:319) ~[?:?] at sun.security.ssl.SSLSocketImpl$AppOutputStream.write(SSLSocketImpl.java:982) ~[?:?] at org.apache.http.impl.io.SessionOutputBufferImpl.streamWrite(SessionOutputBufferImpl.java:124) ~[httpcore-4.4.13.jar!/:4.4.13] at org.apache.http.impl.io.SessionOutputBufferImpl.write(SessionOutputBufferImpl.java:160) ~[httpcore-4.4.13.jar!/:4.4.13] at org.apache.http.impl.io.ContentLengthOutputStream.write(ContentLengthOutputStream.java:113) ~[httpcore-4.4.13.jar!/:4.4.13] at org.apache.http.impl.io.ContentLengthOutputStream.write(ContentLengthOutputStream.java:120) ~[httpcore-4.4.13.jar!/:4.4.13] at org.apache.http.entity.StringEntity.writeTo(StringEntity.java:167) ~[httpcore-4.4.13.jar!/:4.4.13] at org.apache.http.impl.DefaultBHttpClientConnection.sendRequestEntity(DefaultBHttpClientConnection.java:156) ~[httpcore-4.4.13.jar!/:4.4.13] at org.apache.http.impl.conn.CPoolProxy.sendRequestEntity(CPoolProxy.java:152) ~[httpclient-4.5.10.jar!/:4.5.10] at org.apache.http.protocol.HttpRequestExecutor.doSendRequest(HttpRequestExecutor.java:238) ~[httpcore-4.4.13.jar!/:4.4.13] at com.amazonaws.http.protocol.SdkHttpRequestExecutor.doSendRequest(SdkHttpRequestExecutor.java:63) ~[aws-java-sdk-core-1.11.782.jar!/:?] at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:123) ~[httpcore-4.4.13.jar!/:4.4.13] at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) ~[httpclient-4.5.10.jar!/:4.5.10] at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) ~[httpclient-4.5.10.jar!/:4.5.10] at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) ~[httpclient-4.5.10.jar!/:4.5.10] at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) ~[httpclient-4.5.10.jar!/:4.5.10] at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56) ~[httpclient-4.5.10.jar!/:4.5.10] at com.amazonaws.http.apache.client.impl.SdkHttpClient.execute(SdkHttpClient.java:72) ~[aws-java-sdk-core-1.11.782.jar!/:?] at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1330) ~[aws-java-sdk-core-1.11.782.jar!/:?] at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1145) ~[aws-java-sdk-core-1.11.782.jar!/:?]

Java SDK -- OpenJDK 1.11

debora-ito commented 4 years ago

@vk9551 apologies for the super long silence here. Are you still experiencing this issue? Is it intermittent? Or you can reproduce it consistently? Can you share the code where you construct the client?

github-actions[bot] commented 4 years ago

It looks like this issue hasn’t been active in longer than a week. In the absence of more information, we will be closing this issue soon. If you find that this is still a problem, please add a comment to prevent automatic closure, or if the issue is already closed please feel free to reopen it.

vk9551 commented 4 years ago

When you make receiveMessageRequest singleton this issue is happening. I am able to supress this issue bu recreating receive message request for every single call make to sqs

debora-ito commented 4 years ago

Glad you were able to work around it. If you still want us to investigate, could you share a reproducible code?

github-actions[bot] commented 4 years ago

It looks like this issue hasn’t been active in longer than a week. In the absence of more information, we will be closing this issue soon. If you find that this is still a problem, please add a comment to prevent automatic closure, or if the issue is already closed please feel free to reopen it.

bicepjai commented 4 years ago

I see this issue if the file size if large say above 10G