abashev / vfs-s3

Amazon S3 driver for Apache commons-vfs (Virtual File System) project
Apache License 2.0
92 stars 50 forks source link

S3FileObject/S3OutputStream onClose() should setContentLength() #6

Closed leejaywu closed 11 years ago

leejaywu commented 12 years ago

Problem observed:

Truncated uploads w/ non-trivial objects (e.g. missing bytes at the end of a ~230MB object). This has sometimes been but is not always associated with a timeout.

We've worked around this by not using vfs-s3 to upload and instead using jets3t 0.9 directly, but we're not doing much different than vfs-s3 other than specifying the temporary File instead of using a FileChannel and explicitly setting the content length.

Observation -- S3FileObject does not set the content length when you close it (after writing directly to the OutputStream from the FileObject), but the jets3t docs suggest that you must.

http://jets3t.s3.amazonaws.com/toolkit/code-samples.html#uploading

If your data isn't a File or String you can use any input stream as a data source, but you must manually set the Content-Length.