search

krobertson / deb-s3

Easily create and manage an APT repository on S3 -- NO LONGER MAINTAINED
MIT License
482 stars 149 forks source link

`syswrite': Connection reset by peer (Errno::ECONNRESET) #39

Closed ajohnstone closed 8 years ago

ajohnstone commented 10 years ago 
>> Retrieving existing manifests
>> Examining package file checkout-universal-checkout-web-3861-2014-07-02-22-10-10.deb_3861_amd64.deb
/usr/bin/dpkg
>> Uploading packages and new manifests to S3
   -- Transferring pool/c/ch/checkout-universal-checkout-web-3861-2014-07-02-22-10-10.deb_3861_amd64.deb
/usr/lib/ruby/1.9.1/openssl/buffering.rb:317:in `syswrite': Connection reset by peer (Errno::ECONNRESET)
    from /usr/lib/ruby/1.9.1/openssl/buffering.rb:317:in `do_write'
    from /usr/lib/ruby/1.9.1/openssl/buffering.rb:335:in `write'
    from /usr/lib/ruby/1.9.1/net/protocol.rb:199:in `write0'
    from /usr/lib/ruby/1.9.1/net/protocol.rb:173:in `block in write'
    from /usr/lib/ruby/1.9.1/net/protocol.rb:190:in `writing'
    from /usr/lib/ruby/1.9.1/net/protocol.rb:172:in `write'
    from /usr/lib/ruby/1.9.1/net/http.rb:1955:in `send_request_with_body_stream'
    from /usr/lib/ruby/1.9.1/net/http.rb:1921:in `exec'
    from /usr/lib/ruby/1.9.1/net/http.rb:1317:in `block in transport_request'
    from /usr/lib/ruby/1.9.1/net/http.rb:1316:in `catch'
    from /usr/lib/ruby/1.9.1/net/http.rb:1316:in `transport_request'
    from /usr/lib/ruby/1.9.1/net/http.rb:1293:in `request'
    from /usr/lib/ruby/gems/1.9.1/gems/aws-sdk-1.38.0/lib/aws/core/http/connection_pool.rb:340:in `request'
    from /usr/lib/ruby/gems/1.9.1/gems/aws-sdk-1.38.0/lib/aws/core/http/net_http_handler.rb:61:in `block in handle'
    from /usr/lib/ruby/gems/1.9.1/gems/aws-sdk-1.38.0/lib/aws/core/http/connection_pool.rb:129:in `session_for'
    from /usr/lib/ruby/gems/1.9.1/gems/aws-sdk-1.38.0/lib/aws/core/http/net_http_handler.rb:55:in `handle'
    from /usr/lib/ruby/gems/1.9.1/gems/aws-sdk-1.38.0/lib/aws/core/client.rb:252:in `block in make_sync_request'
    from /usr/lib/ruby/gems/1.9.1/gems/aws-sdk-1.38.0/lib/aws/core/client.rb:288:in `retry_server_errors'
    from /usr/lib/ruby/gems/1.9.1/gems/aws-sdk-1.38.0/lib/aws/core/client.rb:248:in `make_sync_request'
    from /usr/lib/ruby/gems/1.9.1/gems/aws-sdk-1.38.0/lib/aws/core/client.rb:510:in `block (2 levels) in client_request'
    from /usr/lib/ruby/gems/1.9.1/gems/aws-sdk-1.38.0/lib/aws/core/client.rb:390:in `log_client_request'
    from /usr/lib/ruby/gems/1.9.1/gems/aws-sdk-1.38.0/lib/aws/core/client.rb:476:in `block in client_request'
    from /usr/lib/ruby/gems/1.9.1/gems/aws-sdk-1.38.0/lib/aws/core/client.rb:372:in `return_or_raise'
    from /usr/lib/ruby/gems/1.9.1/gems/aws-sdk-1.38.0/lib/aws/core/client.rb:475:in `client_request'
    from (eval):3:in `upload_part'
    from /usr/lib/ruby/gems/1.9.1/gems/aws-sdk-1.38.0/lib/aws/s3/multipart_upload.rb:204:in `add_part'
    from /usr/lib/ruby/gems/1.9.1/gems/aws-sdk-1.38.0/lib/aws/s3/s3_object.rb:1722:in `block in write_with_multipart'
    from /usr/lib/ruby/gems/1.9.1/gems/aws-sdk-1.38.0/lib/aws/s3/s3_object.rb:724:in `multipart_upload'
    from /usr/lib/ruby/gems/1.9.1/gems/aws-sdk-1.38.0/lib/aws/s3/s3_object.rb:1721:in `write_with_multipart'
    from /usr/lib/ruby/gems/1.9.1/gems/aws-sdk-1.38.0/lib/aws/s3/s3_object.rb:605:in `write'
    from /usr/lib/ruby/gems/1.9.1/gems/deb-s3-0.6.2/lib/deb/s3/utils.rb:75:in `s3_store'
    from /usr/lib/ruby/gems/1.9.1/gems/deb-s3-0.6.2/lib/deb/s3/manifest.rb:85:in `block in write_to_s3'
    from /usr/lib/ruby/gems/1.9.1/gems/deb-s3-0.6.2/lib/deb/s3/manifest.rb:82:in `each'
    from /usr/lib/ruby/gems/1.9.1/gems/deb-s3-0.6.2/lib/deb/s3/manifest.rb:82:in `write_to_s3'
    from /usr/lib/ruby/gems/1.9.1/gems/deb-s3-0.6.2/lib/deb/s3/cli.rb:141:in `block in upload'
    from /usr/lib/ruby/gems/1.9.1/gems/deb-s3-0.6.2/lib/deb/s3/cli.rb:140:in `each_value'
    from /usr/lib/ruby/gems/1.9.1/gems/deb-s3-0.6.2/lib/deb/s3/cli.rb:140:in `upload'
    from /usr/lib/ruby/gems/1.9.1/gems/thor-0.18.1/lib/thor/command.rb:27:in `run'
    from /usr/lib/ruby/gems/1.9.1/gems/thor-0.18.1/lib/thor/invocation.rb:120:in `invoke_command'
    from /usr/lib/ruby/gems/1.9.1/gems/thor-0.18.1/lib/thor.rb:363:in `dispatch'
    from /usr/lib/ruby/gems/1.9.1/gems/thor-0.18.1/lib/thor/base.rb:439:in `start'
    from /usr/lib/ruby/gems/1.9.1/gems/deb-s3-0.6.2/bin/deb-s3:8:in `<top (required)>'
    from /usr/bin/deb-s3:23:in `load'
    from /usr/bin/deb-s3:23:in `<main>'
leonardehrenfried commented 10 years ago

I'm had the same problem on a Ubuntu 12.04 box (seems fixed in 14.04) and the following blog post had a solution: http://scie.nti.st/2008/3/14/amazon-s3-and-connection-reset-by-peer/

TLDR: Put the following in /etc/sysctl.conf:

# Workaround for TCP Window Scaling bugs in other ppl's equipment:
net.ipv4.tcp_wmem = 4096 16384 512000
net.ipv4.tcp_rmem = 4096 87380 512000

and run sudo sysctl -p.

simonrondelez commented 9 years ago

same issue here. The fix of Lenniboy doesn't help

jhnferraris commented 8 years ago

I'm experiencing this issue in my Ubuntu 14.04. Tried the above fix but no joy.

Here's my logs:

The file size is 50mb

I'm experiencing 2 kinds of errors:

Connection reset

/usr/lib/ruby/1.9.1/net/protocol.rb:199:in `write': Connection reset by peer (Errno::ECONNRESET)
    from /usr/lib/ruby/1.9.1/net/protocol.rb:199:in `write0'
    from /usr/lib/ruby/1.9.1/net/protocol.rb:173:in `block in write'
    from /usr/lib/ruby/1.9.1/net/protocol.rb:190:in `writing'
    from /usr/lib/ruby/1.9.1/net/protocol.rb:172:in `write'
    from /var/lib/gems/1.9.1/gems/aws-s3-0.6.3/lib/aws/s3/extensions.rb:330:in `send_request_with_body_stream'
    from /usr/lib/ruby/1.9.1/net/http.rb:1922:in `exec'
    from /usr/lib/ruby/1.9.1/net/http.rb:1318:in `block in transport_request'
    from /usr/lib/ruby/1.9.1/net/http.rb:1317:in `catch'
    from /usr/lib/ruby/1.9.1/net/http.rb:1317:in `transport_request'
    from /usr/lib/ruby/1.9.1/net/http.rb:1294:in `request'
    from /var/lib/gems/1.9.1/gems/aws-s3-0.6.3/lib/aws/s3/connection.rb:45:in `block in request'
    from /usr/lib/ruby/1.9.1/net/http.rb:746:in `start'
    from /var/lib/gems/1.9.1/gems/aws-s3-0.6.3/lib/aws/s3/connection.rb:52:in `request'
    from /var/lib/gems/1.9.1/gems/aws-s3-0.6.3/lib/aws/s3/base.rb:69:in `request'
    from /var/lib/gems/1.9.1/gems/aws-s3-0.6.3/lib/aws/s3/base.rb:88:in `put'
    from /var/lib/gems/1.9.1/gems/aws-s3-0.6.3/lib/aws/s3/object.rb:241:in `store'
    from /usr/local/bin/lb-backup-db:34:in `<main>'

or it would be a Broken pipe

/usr/lib/ruby/1.9.1/net/protocol.rb:199:in `write': Broken pipe (Errno::EPIPE)
    from /usr/lib/ruby/1.9.1/net/protocol.rb:199:in `write0'
    from /usr/lib/ruby/1.9.1/net/protocol.rb:173:in `block in write'
    from /usr/lib/ruby/1.9.1/net/protocol.rb:190:in `writing'
    from /usr/lib/ruby/1.9.1/net/protocol.rb:172:in `write'
    from /var/lib/gems/1.9.1/gems/aws-s3-0.6.3/lib/aws/s3/extensions.rb:330:in `send_request_with_body_stream'
    from /usr/lib/ruby/1.9.1/net/http.rb:1922:in `exec'
    from /usr/lib/ruby/1.9.1/net/http.rb:1318:in `block in transport_request'
    from /usr/lib/ruby/1.9.1/net/http.rb:1317:in `catch'
    from /usr/lib/ruby/1.9.1/net/http.rb:1317:in `transport_request'
    from /usr/lib/ruby/1.9.1/net/http.rb:1294:in `request'
    from /var/lib/gems/1.9.1/gems/aws-s3-0.6.3/lib/aws/s3/connection.rb:45:in `block in request'
    from /usr/lib/ruby/1.9.1/net/http.rb:746:in `start'
    from /var/lib/gems/1.9.1/gems/aws-s3-0.6.3/lib/aws/s3/connection.rb:52:in `request'
    from /var/lib/gems/1.9.1/gems/aws-s3-0.6.3/lib/aws/s3/base.rb:69:in `request'
    from /var/lib/gems/1.9.1/gems/aws-s3-0.6.3/lib/aws/s3/base.rb:88:in `put'
    from /var/lib/gems/1.9.1/gems/aws-s3-0.6.3/lib/aws/s3/object.rb:241:in `store'
    from /usr/local/bin/lb-backup-db:34:in `<main>'

Anyone got around this?