tweedegolf / storage-abstraction

Provides an abstraction layer for interacting with a storage; the storage can be local or in the cloud.
MIT License
109 stars 18 forks source link

Empty/corrupt file using getFileAsReadable with DigitalOcean Spaces (S3) #16

Closed ochrstn closed 1 year ago

ochrstn commented 3 years ago

Hi!

Firstly, thanks for your work on this project! 🙂

I encountered a problem today while reading a file from a S3 compatible storage (https://www.digitalocean.com/products/spaces/) using getFileAsReadable

The resulting stream was always empty or corrupt (if not specifying end parameter) because the generated Range parameter looked like this:

range=0-undefined

Here is the diff that solved my problem by locally patching the transpiled code:

diff --git a/node_modules/@tweedegolf/storage-abstraction/dist/AdapterAmazonS3.js b/node_modules/@tweedegolf/storage-abstraction/dist/AdapterAmazonS3.js
index 9a8f6b4..be4e1ff 100644
--- a/node_modules/@tweedegolf/storage-abstraction/dist/AdapterAmazonS3.js
+++ b/node_modules/@tweedegolf/storage-abstraction/dist/AdapterAmazonS3.js
@@ -137,7 +137,7 @@ var AdapterAmazonS3 = /** @class */ (function (_super) {
                         params = {
                             Bucket: this.bucketName,
                             Key: fileName,
-                            Range: "bytes=" + options.start + "-" + options.end,
+                            Range: `bytes=${options.start}-${options.end || ""}`
                         };
                         // console.log(`bytes=${options.start}-${options.end}`);
                         return [4 /*yield*/, this.storage.headObject(params).promise()];

I can provide a merge request if you want.

I didn't test if that problem also occurs on AWS S3.

abudaan commented 1 year ago

Thanks! I have added your solution to version 1.4.5