mega-backup on Synology reports "PROGRESS 100%" yet "INCOMPLETE

[rsbrux]

Here is an excerpt from the status output:

-- CURRENT/LAST BACKUP -- FILES UP/TOT FOLDERS CREATED PROGRESS 7060/7061 1039 3.70/3.70 GB 100.00% TAG LOCALPATH REMOT...TPATH STATUS 12090 /volu...p/IMCZ /IMCZ...pricot ACTIVE -- SAVED BACKUPS -- NAME DATE STATUS FILES FOLDERS IMCZ...050000 13Feb2022 05:00:00 INCOMPLETE 6950 996 IMCZ...050000 20Feb2022 05:00:00 INCOMPLETE 7018 1015 IMCZ_...050000 27Feb2022 05:00:00 INCOMPLETE 7060 1039 << How can backup report 100% progress and yet INCOMPLETE? How can I find out whether the backups are really complete? If they aren't, how can I find out what's missing?

[dyuste]

Hi @rsbrux,

The section CURRENT/LAST BACKUP tell us about the last backup run, which in this case completed all the uploads but one: note 7060/7061.

The percentage 100% is based on bytes. An empty file could have failed to upload, causing the confusing percentage.

In order to verify, you can run this command:

mega-transfers --limit=99999 --show-completed | grep -v COMPLETED

Answering your last question, you can explore the list of complete transfers by using the command mega-transfers.

Please note that mega-backup is a BETA functionality, we appreciate and thank you a lot for your feed-back. Your contribution allows us to make it better.

[rsbrux]

@dyuste Thanks for your updated response: it is now a lot clearer. Unfortunately, the latest outputs from -l and -h make it look like nothing is happening:

TAG   LOCALPATH      REMOT...TPATH          STATUS
4     /volu...p/IMCZ /IMCZ...pricot         ACTIVE
  Max Backups:   3
  Period:         "0 0 4 * * 7"
  Next backup scheduled for: Sun, 20 Mar 2022 05:00:00 +0100
   -- CURRENT/LAST BACKUP --
  FILES UP/TOT     FOLDERS CREATED    PROGRESS
       0/0                0         0.00/0.00  B  0.00%
TAG   LOCALPATH      REMOT...TPATH          STATUS
4     /volu...p/IMCZ /IMCZ...pricot         ACTIVE
   -- SAVED BACKUPS --
  NAME           DATE                    STATUS  FILES FOLDERS
  IMCZ_...050000 27Feb2022 05:00:00  INCOMPLETE   7060    1039
  IMCZ_...050000 06Mar2022 05:00:00  INCOMPLETE   7162    1079
  IMCZ_...050000 13Mar2022 05:00:00  INCOMPLETE   7198    1093

The transfers command is interesting, but I would find it much more useful if there were a "--show-incomplete" option. Also, I don't understand why the output of the mega-transfers command you recommend should be piped through "grep -v COMPLETED" when it already has the option "--show-completed".

P.S. I tried the recommended mega-transfers command: mega-transfers --limit=99999 --show-completed | grep -v COMPLETED but got no output at all from it.

[rsbrux]

It gets better! ;-{ Here is the latest output from -l, -h:

TAG   LOCALPATH      REMOT...TPATH          STATUS
4     /volu...p/IMCZ /IMCZ...pricot         ACTIVE
  Max Backups:   1
  Period:         "0 0 4 * * 2"
  Next backup scheduled for: Tue, 22 Mar 2022 05:00:00 +0100
   -- CURRENT/LAST BACKUP --
  FILES UP/TOT     FOLDERS CREATED    PROGRESS
  7211/7212             1106         3.71/3.71 GB  100.00%
TAG   LOCALPATH      REMOT...TPATH          STATUS
4     /volu...p/IMCZ /IMCZ...pricot         ACTIVE
   -- SAVED BACKUPS --
  NAME           DATE                    STATUS  FILES FOLDERS
  IMCZ_...050000 15Mar2022 05:00:00  INCOMPLETE   7211    1106
DIR/SYNC TAG  SOURCEPATH    DESTINYPATH         PROGRESS           STATE
 ⇑      8661 /volu...on.pdf /IMCZ...Events    0.00% of    0.00  B  FAILED

Once again 100% was reported, although evidently one file was missing, but the status of saved backups reported 0% and 0 bytes transferred. Once again, there was no (standard) output from mega-transfers --limit=99999 --show-completed | grep -v COMPLETED A further irritation is that the single backup job specified here wiped out the previously specified job with 3 backups. From the manual I had the impression that multiple backup jobs with different parameters could be configured. IMHO, there is no reason to provide a TAG argument to -lhda if that is not the case. What I want is 1 version each of backups made on Mon.-Sat. and 3 versions of a Sun. backup.

[dyuste]

Hello @rsbrux, We are analyzing your problem and we will come back with a detailed answer soon. My most sincere apologies for the inconvenience.

[rsbrux]

The last few backups appear to have mostly succeeded, despite the confusing messaging. I have spot-checked a few files and the total amount of storage used looks about right. Here is the latest output from

mega-backup -l
mega-backup -h
mega-transfers --limit=99999 --show-completed | grep -v COMPLETED

TAG   LOCALPATH      REMOT...TPATH          STATUS
4     /volu...p/IMCZ /IMCZ...pricot         ACTIVE
  Max Backups:   3
  Period:         "0 0 4 * * 7"
  Next backup scheduled for: Sun, 10 Apr 2022 06:00:00 +0200
   -- CURRENT/LAST BACKUP --
  FILES UP/TOT     FOLDERS CREATED    PROGRESS
  7467/7469             1153         4.13/4.13 GB  100.00%
TAG   LOCALPATH      REMOT...TPATH          STATUS
4     /volu...p/IMCZ /IMCZ...pricot         ACTIVE
   -- SAVED BACKUPS --
  NAME           DATE                    STATUS  FILES FOLDERS
  IMCZ_...050000 20Mar2022 05:00:00  INCOMPLETE   7240    1116
  IMCZ_...060000 27Mar2022 06:00:00  INCOMPLETE   7402    1135
  IMCZ_...060000 03Apr2022 06:00:00  INCOMPLETE   7467    1153
DIR/SYNC TAG  SOURCEPATH    DESTINYPATH         PROGRESS           STATE
 ⇑     75686 /volu...EN.pdf /IMCZ...Events    0.00% of    0.00  B  FAILED
 ⇑     75683 /volu...on.pdf /IMCZ...Events    0.00% of    0.00  B  FAILED

It would be nice if there were a way to identify the failed files/uploads.

[dyuste]

I apologize for the delay in my response. Fortunately our team has been doing great efforts in speeding up the release of MEGAcmd 1.5.0 package. The new version includes improvements on transfers reporting, among many other enhacencements.

The package has already been submitted to Synology and it is currently under test and review. We will keep you tell you as soon as it is approved by Synology and available for download.