search

syncloud / platform

Run popular services on your device with one click
https://syncloud.org
GNU General Public License v3.0
393 stars 40 forks source link

Nextcloud file upload terribly slow over LAN #573

Closed simoneburalli closed 3 years ago

simoneburalli commented 3 years ago

About 6 hrs for 40 GB and about 6000 files (all RAW photos). In my opinion it is too slow, almost unusable. Any idea to start from to understand what's wrong with the system configuration? Nextcloud has a slow responsiveness also in web interface to navigate dashboard, applications, etc. Any help will be appreciated a lot. Thanks in advance

image

cyberb commented 3 years ago

What is the device (cpu)? How much RAM (total/available)?

cyberb commented 3 years ago

PHP is not the fastest language on the planet, so you need more resources. If you have a lot files to transfer, one option is to copy directly to disk (attach) or to use sftp (to /data/nextcloud/[user]/files) and then let Nextcloud know about it: https://github.com/syncloud/platform/wiki/Nextcloud#file-scan

simoneburalli commented 3 years ago

What is the device (cpu)? How much RAM (total/available)?

It's an Odtoid HC4 (8GB RAM, if I well remember) with 2 HDD's in RAID1. I'll try sftp. Thanks a lot

UPDATE: through sftp the transfer rate is about 60 times faster (5 minutes instead of 5 hrs). Thanks a lot

UPDATE#2: the entire process with almost a double amount of data that I was trying to transfer yesterday took less than 2 hrs (snap run nextcloud.occ files:scan --all took only 45 mins, see below). Some details: I have transferred 130 GB distributed over 30k files . @cyberb I think it would be great to put this solution in evidence for first time users that normally need to perform an initial massive upload of data before starting to use the sync client. Here follows a small guide for dummies, as am I:

  1. to connect via sftp to the Syncloud machine from Windows, download WinSCP. (Take into account the opportunity to buy it from Windows store to support its development);
  2. perform an installation of the same program by using all default options;
  3. open a new connection with the IP of your Syncloud machine, user and pwd. You can also save it (optional) in case you plan to reuse it frequently. Keep all defaults parameters, like port and so on;
  4. transfer the files you want to /data/nextcloud/[user]/files (example: /data/nextcloud/Maryann/files);
  5. after the transfer has come to an end, pay attention on WinSCP that the folder(s) that you have just uploaded has the right owner: it must be "nextcloud". Probably you will see "root" as the owner. To change the owner you have to run chown recursively (option "-R") on the folder(s) you have transferred. Example: $ chown -R nextcloud:nextcloud /opt/disk/external/nextcloud/Maryann/files/My_Uploaded_Folder. To perform this operation open the console of Syncloud (download Putty to avoid the connection of both a physical monitor and keyboard) and login with your credentials;
  6. again on the console: snap run nextcloud.occ files:scan --all. This will make Nextcloud aware of the changes you have performed on disk;
  7. Leave Putty open while you wait for the completion of the process. It will take some time according to the amount of transferred data. At the end, the process returns this:

image

Ciao

@cyberb to me you can close this issue. Thanks a lot