Slow file uploading after meteor upgrade

[valerymagadeeva]

Hello! I updated my meteor application to meteor 2.7 (and meteor-files to 2.3.0) and as a result I have a strange issue - the file uploading takes too long time, it uploads extremely slow. (the difference between the old version and the new one is about 30 seconds for 40mb file). I tried to use both of DDP and HTTP transports and nothing helps. I noticed that when I used HTTP to upload files the meteor server response took at least a second for every chunk. So I guess the problem is somewhere between the client side and the server side. To upload file from client i use insert() method with chunkSize equals to "dynamic" value. The server side code is the FilesCollection constructor:

const Uploads = new FilesCollection({
    collectionName: 'preupload',
    allowClientCode: false,
    debug: false,
    protected: true,
    onBeforeUpload(file) {},
    onAfterUpload(uploadedFile) {}
})

• Meteor-Files version - 2.3.0;
• Meteor version - 2.7;
• Browser/OS - Google Chrome/MacOS;
• Seems like that might be server side issue;

Client logs: [FilesCollection] [insert()]

vue-composition-api.mjs:2246 [FilesCollection] [FileUpload] [constructor]
vue-composition-api.mjs:2246 onBeforeUpload => Mon Sep 26 2022 13:38:11 GMT+0400 (Samara Standard Time)
vue-composition-api.mjs:2246 [FilesCollection] [insert] using WebWorkers
upload.js:368 loadFile ccsetup592.bin (1).zip: 95912.81884765625 ms
vue-composition-api.mjs:2246 [FilesCollection] [UploadInstance] [sendEOF] false
vue-composition-api.mjs:2246 [FilesCollection] [UploadInstance] [end] ccsetup592.bin (1).zip
upload.js:319 insert ccsetup592.bin (1).zip: 96609.42993164062 ms

Server logs(last part) (do not pay attention to warning, i just need console.time for time measurements):

I20220926-09:39:42.742(0)? [FilesCollection] [Upload] [DDP] Got #141/149 chunks, dst: ccsetup592.bin (1).zip
I20220926-09:39:42.742(0)? onBeforeUpload => 2022-09-26T09:39:42.742Z
W20220926-09:39:42.743(0)? (STDERR) (node:129) Warning: Label 'file uploading1' already exists for console.time()
I20220926-09:39:43.262(0)? [FilesCollection] [Upload] [DDP] Got #142/149 chunks, dst: ccsetup592.bin (1).zip
I20220926-09:39:43.263(0)? onBeforeUpload => 2022-09-26T09:39:43.262Z
W20220926-09:39:43.263(0)? (STDERR) (node:129) Warning: Label 'file uploading1' already exists for console.time()
I20220926-09:39:43.888(0)? [FilesCollection] [Upload] [DDP] Got #143/149 chunks, dst: ccsetup592.bin (1).zip
I20220926-09:39:43.888(0)? onBeforeUpload => 2022-09-26T09:39:43.888Z
W20220926-09:39:43.888(0)? (STDERR) (node:129) Warning: Label 'file uploading1' already exists for console.time()
I20220926-09:39:44.825(0)? [FilesCollection] [Upload] [DDP] Got #144/149 chunks, dst: ccsetup592.bin (1).zip
I20220926-09:39:44.826(0)? onBeforeUpload => 2022-09-26T09:39:44.825Z
W20220926-09:39:44.826(0)? (STDERR) (node:129) Warning: Label 'file uploading1' already exists for console.time()
I20220926-09:39:45.649(0)? [FilesCollection] [Upload] [DDP] Got #145/149 chunks, dst: ccsetup592.bin (1).zip
I20220926-09:39:45.649(0)? onBeforeUpload => 2022-09-26T09:39:45.649Z
W20220926-09:39:45.650(0)? (STDERR) (node:129) Warning: Label 'file uploading1' already exists for console.time()
I20220926-09:39:46.173(0)? [FilesCollection] [Upload] [DDP] Got #146/149 chunks, dst: ccsetup592.bin (1).zip
I20220926-09:39:46.174(0)? onBeforeUpload => 2022-09-26T09:39:46.173Z
W20220926-09:39:46.174(0)? (STDERR) (node:129) Warning: Label 'file uploading1' already exists for console.time()
I20220926-09:39:46.954(0)? [FilesCollection] [Upload] [DDP] Got #147/149 chunks, dst: ccsetup592.bin (1).zip
I20220926-09:39:46.955(0)? onBeforeUpload => 2022-09-26T09:39:46.954Z
W20220926-09:39:46.955(0)? (STDERR) (node:129) Warning: Label 'file uploading1' already exists for console.time()
I20220926-09:39:47.503(0)? [FilesCollection] [Upload] [DDP] Got #148/149 chunks, dst: ccsetup592.bin (1).zip
I20220926-09:39:47.503(0)? onBeforeUpload => 2022-09-26T09:39:47.503Z
W20220926-09:39:47.504(0)? (STDERR) (node:129) Warning: Label 'file uploading1' already exists for console.time()
I20220926-09:39:48.121(0)? [FilesCollection] [Upload] [DDP] Got #149/149 chunks, dst: ccsetup592.bin (1).zip
I20220926-09:39:48.121(0)? onBeforeUpload => 2022-09-26T09:39:48.121Z
W20220926-09:39:48.121(0)? (STDERR) (node:129) Warning: Label 'file uploading1' already exists for console.time()
I20220926-09:39:48.192(0)? [FilesCollection] [Upload] [DDP] Got #-1/149 chunks, dst: ccsetup592.bin (1).zip
I20220926-09:39:48.193(0)? onBeforeUpload => 2022-09-26T09:39:48.192Z
W20220926-09:39:48.193(0)? (STDERR) (node:129) Warning: Label 'file uploading1' already exists for console.time()
I20220926-09:39:48.195(0)? [FilesCollection] [Upload] [finish(ing)Upload] -> assets/app/uploads/preupload/sQW9M6qt4WaLbzefJ.zip
I20220926-09:39:48.211(0)? [FilesCollection] [Upload] [finish(ed)Upload] -> assets/app/uploads/preupload/sQW9M6qt4WaLbzefJ.zip
I20220926-09:39:48.212(0)? file uploading1: 1:36.508 (m:ss.mmm)

Thank you in advance!

[dr-dimitru]

Hello @valerymagadeeva ,

Please update your OP following our ISSUE_TEMPLATE. And post logs with debug enabled

[valerymagadeeva]

@dr-dimitru updated!

[dr-dimitru]

@valerymagadeeva thanks, let's start with question:

1. Why you upload ccsetup592.bin but file saved as ZIP (sQW9M6qt4WaLbzefJ.zip)?

[valerymagadeeva]

The file's full name is ccsetup592.bin (1).zip - it's zip archive and it should have such extension. Anyway I tried to upload a binary file - the result is the same

[dr-dimitru]

@valerymagadeeva Do you zip the file during or after upload?

[dr-dimitru]

@valerymagadeeva to make sure this isn't caused your development environment please download, launch locally, and test uploads via this app (runs on the latest meteor and meteor-files)

[valerymagadeeva]

@dr-dimitru I unzip the file after uploading in the other part of our system. The problem is not connected to zip file. I tried your demo app and got the same result

[dr-dimitru]

@valerymagadeeva do you get a good speed when uploading at files.veliov.com ?

[valerymagadeeva]

@dr-dimitru nope, almost the same result (maybe 10 seconds faster). While in the old version of meteor I have uploaded the file for 45-50 seconds (vs 1m 20 - 1m30 seconds in your demo and my app with the new version of meteor)

[dr-dimitru]

@valerymagadeeva loads very local, any other device to test on?

I'm getting 10MB/s, which is above my upload speed test

[valerymagadeeva]

I have lower speed when trying to upload via your demo app. But my internet speed is higher

[dr-dimitru]

1. But you're saying speed was higher before last update? Including on demo website?
2. What speed do you have locally?

[valerymagadeeva]

1. idk if the speed was higher on your demo website. Now i have two git branches with file uploading - the first one with old Meteor version (1.11 unfortunately and the file is uploading for 40-50 seconds) and the second one with the new version.
2. What do you mean under local speed? My hard drive write speed?

[dr-dimitru]

TLTD; we need clear reproduction steps in order to identify an issue

[valerymagadeeva]

What info can i provide you?

[dr-dimitru]

@valerymagadeeva can not come up with something specific, as I'm not sure how to localize your issue

[valerymagadeeva]

@dr-dimitru ok, I'll try to provide something specific to reproduce the issue. But it seems I have a poor upload speed (according to the information from the demo app). Idk why and im sure it's not connected to my internet or something

[dr-dimitru]

Does anyone else experiencing this? Or have ever experienced?