When you issue 2 put_file request with different file size, the header content-length is stuck at the size of the first file, causing a 400 return code from server due to the invalid content-length.
To reproduce the issue :
f_path = 'tmp'
with open(f_path, 'wb') as f:
f.write("toto")
s3.put_file(f_path, f_path)
with open(f_path, 'wb') as f:
f.write("toto2")
s3.put_file(f_path, f_path)
class StreamingMixin(object):
def put_file(self, key, fp, acl=None, metadata={}, progress=None,
size=None, mimetype=None, transformer=None, headers=None):
"""Put file-like object or filename *fp* on S3 as *key*.
*fp* must have a read method that takes a buffer size, and must behave
correctly with regards to seeking and telling.
*size* can be specified as a size hint. Otherwise the size is figured
out via ``os.fstat``, and requires that *fp* have a functioning
``fileno()`` method.
*progress* is a callback that might look like ``p(current, total,
last_read)``. ``current`` is the current position, ``total`` is the
size, and ``last_read`` is how much was last read. ``last_read`` is
zero on EOF.
"""
if not headers:
headers = {}
do_close = False
Hi,
When you issue 2 put_file request with different file size, the header content-length is stuck at the size of the first file, causing a 400 return code from server due to the invalid content-length.
To reproduce the issue :
Tested against RADOS GATEWAY.
The root cause is due to the fact that is header param is empty when calling put_file, a default dict is assigned in the function definition. https://github.com/lericson/simples3/blob/master/simples3/streaming.py#L34
As a result, every subsequent call will get the same object reference (as the dict is in the class scope and not function scope).
Content-length will always be set after the first call, as a result it is not updated https://github.com/lericson/simples3/blob/master/simples3/streaming.py#L58
Here is a fix :