TTLabs / EvaporateJS

Javascript library for browser to S3 multipart resumable uploads
1.82k stars 207 forks source link

upload speed limited to ~2MB/s #400

Open ddereszewski opened 6 years ago

ddereszewski commented 6 years ago

I set up the plugin, and it uploads the files correctly, but slow. I played with partSize param but cannot make it faster. My upload speed on the internet tests is around 20Mb/s

Any tips for that?

this is my config

const uploadWeb = (emitter,file, url, mime_type, formInputs) => {

        const config = {
            signerUrl:  BaseApi.baseUrl + 'mediaSignUpload',
            signHeaders: {
                'Authorization' : BaseApi.authToken
            },
            aws_key: 'MYAWSKEY',
            bucket: 'MYBUCKET',
            logging: true,
          //  partSize: 20 * 1024 * 1024, // 100MB
            awsSignatureVersion: '4',
            awsRegion: 'eu-west-1',
            progressIntervalMS: 5000,
            cryptoHexEncodedHash256: function (data) { return AWS.util.crypto.sha256(data, 'hex'); },
            cloudfront: true,
            computeContentMd5: true,
            cryptoMd5Method: function (data) { return AWS.util.crypto.md5(data, 'base64'); },
        };

        Evaporate.create(config)
            .then(function (evaporate) {
                const addConfig = {
                    name: formInputs.key,
                    file: file,
                    xAmzHeadersAtInitiate: {'x-amz-acl': 'private'},
                    progress: function (p, stats) {
                        const progress = parseFloat(p * 100).toFixed(1);
                        emitter({ progress, stats });
                    },

                };
                evaporate.add(addConfig)
                    .then(function (awsObjectKey) {
                            emitter({success: awsObjectKey});
                            emitter(END);
                        },
                        function (reason) {
                            emitter({err: reason});
                            emitter(END);
                            //console.log('File did not upload sucessfully:', reason);
                        });
            });
    }
HemalR commented 6 years ago

Did you get this resolved? Just out of interest?

ddereszewski commented 6 years ago

I never got faster than 5mb but I guess it could be limitation of the upload endpoint, I played with various parameters including chunk sizes but nothing helped.

I also realized I had a logic problem as I was calling Evaporate.create multiple times for multiple files which results in huge memory usage and speed issues.

DJSab commented 5 years ago

On speedtest.net I got ~60mb of uploading speed but it never go up more than 8mb/s when uploading files. I'm using same config as from the example from repository and even changing partSize wouldn't help.

jakubzitny commented 5 years ago

Have you guys tried uploading from another AWS library, just for comparison? Try aws-sdk to see if you can get faster?