uploading file to s3 results in error 104 (connection reset)

[mjpan]

this is on boto 2.27.0. uploading a 14 byte file works, but 512k file causes this error. using aws cli to upload works on the 512k file

File "/usr/local/lib/python2.7/dist-packages/boto-2.27.0-py2.7.egg/boto/s3/key.py", line 1315, in set_contents_from_filename encrypt_key=encrypt_key) File "/usr/local/lib/python2.7/dist-packages/boto-2.27.0-py2.7.egg/boto/s3/key.py", line 1246, in set_contents_from_file chunked_transfer=chunked_transfer, size=size) File "/usr/local/lib/python2.7/dist-packages/boto-2.27.0-py2.7.egg/boto/s3/key.py", line 725, in send_file chunked_transfer=chunked_transfer, size=size) File "/usr/local/lib/python2.7/dist-packages/boto-2.27.0-py2.7.egg/boto/s3/key.py", line 914, in _send_file_internal query_args=query_args File "/usr/local/lib/python2.7/dist-packages/boto-2.27.0-py2.7.egg/boto/s3/connection.py", line 633, in make_request retry_handler=retry_handler File "/usr/local/lib/python2.7/dist-packages/boto-2.27.0-py2.7.egg/boto/connection.py", line 1046, in make_request retry_handler=retry_handler) File "/usr/local/lib/python2.7/dist-packages/boto-2.27.0-py2.7.egg/boto/connection.py", line 919, in _mexe request.body, request.headers) File "/usr/local/lib/python2.7/dist-packages/boto-2.27.0-py2.7.egg/boto/s3/key.py", line 815, in sender http_conn.send(chunk) File "/usr/lib/python2.7/httplib.py", line 794, in send self.sock.sendall(data) File "/usr/lib/python2.7/ssl.py", line 229, in sendall v = self.send(data[count:]) File "/usr/lib/python2.7/ssl.py", line 198, in send v = self._sslobj.write(data) error: [Errno 104] Connection reset by peer

[fbeister]

Just wanted to say "me too". I use boto through duplicity to access S3. On another machine, I have boto 2.2.2 and it works there. After downgrading boto from 2.27 to 2.2.2, the error persists, so I suspect the problem to lie either in ssl.py or the OS (Ubuntu Linux 14.04)

[fbeister]

Found a solution (at least for duplicity): https://github.com/aws/aws-cli/issues/634

instead of "s3+http://bucketname" use "s3:///s3.amazonaws.com/bucketname"

[danielgtaylor]

From the CLI issue:

I'm reopening this issue. After debugging this I can confirm what others have said. This problem exists when trying to upload a large file to a newly created bucket that's not in the classic region.

From what I can tell the CLI is properly retrying requests and following 307 redirects. The problem, however, is that the CLI sends the entire request and then waits for a response. However, S3 will immediately send the 307 response before we've finished sending the body. Eventually it will just close the connection, and if we're still in the process of streaming the body, we will not see the response. Instead we get the ConnectionError as shown in the various debug logs above.

The normal way to address this would be to use the expect 100 continue header. However, the HTTP client we use (requests) does not support this. There might be a way to work around this, but I'll need to do some digging into the requests library to see the best way to fix this issue.

This will require adding some support for currently unsupported HTTP headers.

[avandendorpe]

I think the fix for awscli addresses this as it was merged in botocore. @jamesls - is that correct?

[danielgtaylor]

The fix for Botocore was in https://github.com/boto/botocore/commit/9e59c4e0e69ca6535dde35d839e6e5e65e03a5bb and seems to have solved the issue.

We need to backport it to Boto. I'll see about trying to get it into the next release.

[j0hnsmith]

I'm seeing a lot of these errors, please backport the fix asap.

[unwitting]

@danielgtaylor any update on getting it ported back into boto?

[jvantuyl]

+1

[anna-buttfield-sirca]

This issue hit us too. The workaround that worked for us was explicitly connecting to the bucket's region. Because we can't guarantee which region buckets are in we use this code:

conn = boto.connect_s3()
bucket = conn.get_bucket(bucket_name)
bucket_location = bucket.get_location()
if bucket_location:
    conn = boto.s3.connect_to_region(bucket_location)
    bucket = conn.get_bucket(bucket_name)

[charles-vdulac]

+1

[gpg90]

+1

[whitequark]

+1

[dwijnand]

:+1:

[eberle1080]

+1

[wengole]

This doesn't quite work as regions and locations have different names, however the concept does help (connecting to the correct region)

[fangpenlin]

+1

[volnt]

+1

[hwkns]

+1

[antonagestam]

Come on ppl, commenting +1 isn't helping anyone.

[hwkns]

@antonagestam, my intent was to help provide a sense of the number of people affected by this bug, which should hopefully lend it some priority. If there's something more I can do to help, please let me know. This is a sponsored project now, and this issue has been around for at least 9 months already, affecting quite a few users. I'm just trying not to let it languish.

[hellwolf]

+1

[ledmonster]

+1

[spikewilliams]

I am hitting this bug. Just as someone referenced above, I am also on Ubuntu Linux 14.04.

[matiasinsaurralde]

+1

[sebastibe]

+1

[vperron]

+1

[wvengen]

+1

[alfetopito]

If it helps anything, +1 (Ubuntu 14.04 x64 on EC2, Python 3.4.2, boto==2.36.0)

[cy-yoshikazu-umino]

+1

[nachobitpagos]

Hey @danielgtaylor any news on these?

[kageurufu]

+1

[noahmatisoff]

+1

[luster]

+1

[andtrott]

+1

[safetythird]

+1

[urbanski]

+1

[galuszkak]

+1 from me...

[RobertBiehl]

+1 Is it still the same issue as one year ago?

[raianul]

hi, I am facing this problem for boto 2.36.0, does any one have any solution ?

[badmetacoder]

Any chance of getting it fixed anytime soon?

[jforest]

+1 seeing it in 2.37

[vperron]

Disagreed. This bug is still a blocker to us.

[galuszkak]

@matisoffn please include fact that Open source projects like boto are done by community. You are part of it, so... ;) Pull Request are always welcome :+1: (I've never been not welcomed for submiting patches so this is from my experience). I've personally encourage you to submit fix for this one if you have time :+1: :dancer: .

[disruptek]

@galuszkak your reasoning is sound except for the fact that this project is too heavily saturated with issues for the maintainers to even execute merges on perfectly acceptable pull requests. If the problem doesn't exist in botocore, it seems unlikely that it will be fixed here.

[antonagestam]

@disruptek so fork it.

[unwitting]

@antonagestam why would @disruptek forking off help? Instead of a heavily saturated repo here with a reasonably solid group of people behind it, he'd be taking on the same set of problems as an individual - forking doesn't create more people to look at the issues.

[galuszkak]

@unwitting only from beginning of 2015 there were merged 30 Pull Requests. So it's really active project. Forking and submitting Pull requests is actually doing something more valuable than complaining and requesting to close issue just because none is picking that issue to fix. It really doesn't matter if this is done by one person or group. For me it's always about joy of giving something back to community.

[unwitting]

@galuszkak if the stats are that this is an active project where PRs are accepted then my argument doesn't stand - I was working off an assumption based on @disruptek's comment, and I'm glad you had a more reasoned response than "so fork it" which doesn't exactly say "actually, no, it's fine and welcome, please go ahead". Comment fully rescinded, and the joy of giving something back fully agreed with :)

[ryan-lane]

+1 this is an incredibly annoying bug.

[ccemeraldeyes]

+1