s3tools / s3cmd

Official s3cmd repo -- Command line tool for managing S3 compatible storage services (including Amazon S3 and CloudFront).
https://s3tools.org/s3cmd
GNU General Public License v2.0
4.54k stars 904 forks source link

when deleting files [Errno 104] Connection reset by peer #1346

Open cwjjun opened 10 months ago

cwjjun commented 10 months ago

When I use the command ‘s3cmd rm --recursive’ to delete a large number of sub-files under a folder, the following problem occurs.

S3cmd: 2.3.0 python: 2.7.16 (default, Oct 10 2019, 22:02:15) [GCC 8.3.0] environment LANG=zh_CN

Traceback (most recent call last): File "/usr/local/bin/s3cmd", line 3286, in rc = main() File "/usr/local/bin/s3cmd", line 3183, in main rc = cmd_func(args) File "/usr/local/bin/s3cmd", line 712, in cmd_object_del rc = subcmd_batch_del_iterative(uri_str = uri_str) File "/usr/local/bin/s3cmd", line 738, in subcmd_batch_del_iterative response = s3.object_batch_delete_uri_strs([uri.compose_uri(bucket, item['Key']) for item in to_delete]) File "/usr/local/lib/python2.7/dist-packages/S3/S3.py", line 803, in object_batch_delete_uri_strs response = self.send_request(request) File "/usr/local/lib/python2.7/dist-packages/S3/S3.py", line 1525, in send_request response["data"] = http_response.read() File "/usr/lib/python2.7/httplib.py", line 583, in read return self._read_chunked(amt) File "/usr/lib/python2.7/httplib.py", line 641, in _read_chunked value.append(self._safe_read(chunk_left)) File "/usr/lib/python2.7/httplib.py", line 698, in _safe_read chunk = self.fp.read(min(amt, MAXAMOUNT)) File "/usr/lib/python2.7/socket.py", line 384, in read data = self._sock.recv(left) error: [Errno 104] Connection reset by peer

rampageservices commented 9 months ago

Is this repeatable? Do you need to use Python 2.7.16?

lavigne958 commented 5 months ago

This is transient error, it means the server closed the connection. Cloud for many reasons (very slow connection, connection interruption, etc).

it has nothing to do with s3cmd and s3cmd can't do anything about it, the connection instantly stopped.