Open vmarimuthu opened 7 years ago
Does anyone have a workaround for this issue?
I'm able to download files using client.downloadFile
, but uploading using client.uploadFile
or client.uploadDir
gives an error.
Either: NoSuchBucket: The specified bucket does not exist
(which is weird, because downloadFile
works with the same params)
Or: Error: Non-file stream objects are not supported with SigV4
Since my app was also using serverless I switched to using the serverless s3 client package and it works outside of us-east-1
(e.g. the much newer us-east-2
).
This is solved by updating the aws-sdk
package. It works for me (eu-west-2 region) when I updated the aws-sdk
package to v2.100.0
.
Here's a workaround: Use the S3 Client from the latest SDK and pass it on to the s3
client
const AWS = require('aws-sdk');
const s3 = require('s3');
const awsS3Client = new AWS.S3({
region: 'eu-central-1',
signatureVersion: 'v4'
});
const client = s3.createClient({
s3Client: awsS3Client
});
@LukasBombach still works 👍
Ran into the SigV4 issue using the webpack-s3-plugin
and resorted to adding an upload script with this workaround.
Other folks stop just short of a full answer here but not including how to specify credentials in the AWS.S3 client, here's the full version that you'll need:
const AWS = require('aws-sdk');
const s3 = require('s3');
...
const awsS3Client = new AWS.S3({
region: process.env.AWS_ZONE,
accessKeyId: process.env.AWS_ACCESS_KEY_ID,
secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
signatureVersion: 'v4',
});
const client = s3.createClient({
s3Client: awsS3Client
});
@LukasBombach Thank you sir, you've saved my day.
@LukasBombach , @ryanostrom I was also facing the same issue (Error: Non-file stream objects are not supported with SigV4 in AWS.S3) but after updating as per your suggestion I m getting an timeout error somehow. After 100% its end event does not get fired. Thanks in advance. As I m new to this so may be I am missing something here.
Here is the details : "aws-sdk": "^2.478.0", "s3": "^4.4.0",
const awsS3Client = new AWS.S3({ accessKeyId: argv.accessKeyId, secretAccessKey: argv.secretAccessKey, region: 'us-east-2', signatureVersion: 'v4' });
const client = s3.createClient({ s3Client: awsS3Client });
let uploaderObj;
uploaderObj = client.uploadDir(params);
let prevProgress = 0;
uploaderObj.on('progress', function () {
if (prevProgress !== uploaderObj.progressAmount) {
console.log(`Progress: ${Math.round(uploaderObj.progressAmount / uploaderObj.progressTotal * 100)}%`);
prevProgress = uploaderObj.progressAmount;
}
});
uploaderObj.on('error', (error) => {
reject(error);
});
uploaderObj.on('end', () => {
if (prevProgress === 0)
console.log('Nothing to upload. Already up to date.');
resolve('Done uploading.');
})
@manoj-compro me too
@manoj-compro me too, same version of sdk and s3.
@omarandrade @manoj-compro https://github.com/andrewrk/node-s3-client/issues/223
@manoj-compro Even i am facing same issue , after 100% it is getting timeed out . the folder is not uploading to S3 . Any fix for this
require('@auth0/s3') worked for me with same API
Bro I did not understood , where i need to use this require('@auth0/s3') , can u share me the complete code . it will be helpfull
uninstall S3 and install https://www.npmjs.com/package/@auth0/s3 this module as this is an extension of s3 and issue is fixed in @auth0/s3. All other functions/API are same.
in Js file replace require('s3') by require('@auth0/s3')
Thanks bro , Now there is no error thrown . now i am getting "done uploading" but the folder is not getting synced , or uploaded to S3 , here is my code . please check once . help me out this .
const AWS = require('aws-sdk'); const s3 = require('@auth0/s3');
const awsS3Client = new AWS.S3( { accessKeyId: accessKeyId, secretAccessKey: secretAccessKey, region: 'ap-south-1', signatureVersion: 'v4' });
const client = s3.createClient({ s3Client: awsS3Client } );
console.log( 'my', mybucket1);
var params = {
localDir: './' + local1 ,
deleteRemoved: false,
s3Params: {
Bucket: mybucket1,
Prefix: "/prefix",
ACL: 'public-read'
},
};
var uploader = client.uploadDir(params);
uploader.on('error', function(err) {
console.error("unable to sync:", err.stack);
});
uploader.on('progress', function() {
console.log("progress", uploader.progressAmount, uploader.progressTotal);
});
uploader.on('end', function() {
console.log( "done uploading" );
})
Nice work Bro
@manoj-compro help me on the above code shared .. now i am getting error as "unable to sync: KMS.NotFoundException: Invalid arn"****
Below is the working code. if still not fix then try by changing region of bucket.
var s3 = require('@auth0/s3');
var s3Client = s3.createClient({
s3Options: {
accessKeyId: argv.accessKeyId
secretAccessKey: argv.secretAccessKey
region: 'us-east-2'
}
});
const params = {
localDir: './dist',
deleteRemoved: true,
s3Params: {
Bucket: config[argv.env].s3Bucket,
Prefix: "",
},
};
uploaderObj = s3Client.uploadDir(params);
uninstall S3 and install https://www.npmjs.com/package/@auth0/s3 this module as this is an extension of s3 and issue is fixed in @auth0/s3. All other functions/API are same.
in Js file replace require('s3') by require('@auth0/s3')
This solved the issue for us.
Can you upgrade aws-sdk version?