S3 implementation of `Drive.putStream()` missing required header

[thecodemill]

The S3 implementation of Drive.putStream() is currently broken.

S3 requires a ContentLength header and manually adding one fixes the issue. However, as we can't know the length of a readable stream, this results in "E_CANNOT_WRITE_FILE: Cannot write file at location" errors by default. It would be great if Drive could handle this automatically, eg. the approach outlined at https://github.com/aws/aws-sdk-go/issues/122#issuecomment-76826724. Failing that, an update to the documentation to explain the limitation would be appreciated.

Thanks

---

References:

• Issue #2348: [S3 Client] Errors if the Body of PutObjectCommand is a Readable type
• Official S3 upgrading guide for Multipart Upload in SDK v3

[leonardomarciano]

+1

[capoia]

@thecodemill did you manage to solve this problem? is there any way to set the content length manually?

[thecodemill]

@capoia No, in the end I made a change to our business logic to store the file size (amongst other metadata properties) at the time we receive the file. This size can then be referenced when streaming to S3. The process was already writing a db record, so adding a few extra columns actually turned out to be a bonus 👍

[capoia]

@capoia No, in the end I made a change to our business logic to store the file size (amongst other metadata properties) at the time we receive the file. This size can then be referenced when streaming to S3. The process was already writing a db record, so adding a few extra columns actually turned out to be a bonus +1

But how do you upload (using the Drive that adonis provides) the file size? I know using other libraries, but using Drive.putStream, how do I send the content-length?

[thecodemill]

Third argument of putStream() accepts an array of additional headers. Just send it as a “Content-Length” header.

Sent from my iPhone

On 22 Dec 2021, at 21:02, capoia @.***> wrote:

 @capoia No, in the end I made a change to our business logic to store the file size (amongst other metadata properties) at the time we receive the file. This size can then be referenced when streaming to S3. The process was already writing a db record, so adding a few extra columns actually turned out to be a bonus +1

But how do you upload (using the Drive that adonis provides) the file size? I know using other libraries, but using Drive.putStream, how do I send the content-length?

— Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you were mentioned.

[capoia]

Third argument of putStream() accepts an array of additional headers. Just send it as a “Content-Length” header. … Sent from my iPhone On 22 Dec 2021, at 21:02, capoia @.***> wrote:  @capoia No, in the end I made a change to our business logic to store the file size (amongst other metadata properties) at the time we receive the file. This size can then be referenced when streaming to S3. The process was already writing a db record, so adding a few extra columns actually turned out to be a bonus +1 But how do you upload (using the Drive that adonis provides) the file size? I know using other libraries, but using Drive.putStream, how do I send the content-length? — Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you were mentioned.

I don't know if this has changed recently but the third option is of type ContentHeaders, and it doesn't include contentlenght.

[thetutlage]

The ContentHeaders type has a list of whitelisted properties that is applicable across all the drivers. However, you can still send additional headers accepted by s3.

{
  "Content-Length": theValue
}

@thecodemill Sorry, I haven't been able to pay much attention to this issue. I will go through your shared links and will act accordingly