One SD Card won't backup

ruebyi commented 3 years ago

I'm just on a trip and I'm using LBB on a pi4 to Backup 4 Cameras.

4 of 5 Cards are working properly, one not...

That's what the log gives to me:

2021/07/31 08:22:27 [5920] building file list 2021/07/31 08:22:27 [5920] rsync: chown "/media/storage/202107310002-27285/." failed: Operation not permitted (1) 2021/07/31 08:22:27 [5920] .d..tpog... ./ 2021/07/31 08:22:27 [5920] Number of files: 1 (dir: 1) 2021/07/31 08:22:27 [5920] Number of created files: 0 2021/07/31 08:22:27 [5920] Number of deleted files: 0 2021/07/31 08:22:27 [5920] Number of regular files transferred: 0 2021/07/31 08:22:27 [5920] Total file size: 0 bytes 2021/07/31 08:22:27 [5920] Total transferred file size: 0 bytes 2021/07/31 08:22:27 [5920] Literal data: 0 bytes 2021/07/31 08:22:27 [5920] Matched data: 0 bytes 2021/07/31 08:22:27 [5920] File list size: 0 2021/07/31 08:22:27 [5920] File list generation time: 0.001 seconds 2021/07/31 08:22:27 [5920] File list transfer time: 0.000 seconds 2021/07/31 08:22:27 [5920] Total bytes sent: 55 2021/07/31 08:22:27 [5920] Total bytes received: 110 2021/07/31 08:22:27 [5920] sent 55 bytes received 110 bytes 330.00 bytes/sec 2021/07/31 08:22:27 [5920] total size is 0 speedup is 0.00 2021/07/31 08:22:27 [5920] rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1207) [sender=3.1.3] Delete log

No ID File ist written.

The SD Card is from a DJI Mini 2 - DCIM Folder is in the main Directory, no Writeprotection etc. it's exFAT as all of my SD Cards.

The Same Card from my GoPro Backups perfectly.

dmpop commented 3 years ago

No idea what causes the problem. Perhaps something to do with permissions? Stefan has fixed a few problems related to sudo in the backup script. Can you try the latest version?

ruebyi commented 3 years ago

Seems to work, but let me verify in hohe next days

dmpop commented 3 years ago

Excellent! Feel free to close the issue if the solution works for you. 😃

ruebyi commented 3 years ago

So it happened again.

First attempted worked perfect. Two days later - a Card that already worked can‘t be backuped anymore…

ruebyi commented 3 years ago

2021/08/01 23:01:39 [4851] rsync: chown "/media/storage/202108012301-31863/." failed: Operation not permitted (1) 2021/08/01 23:01:39 [4851] .d...pog... ./ 2021/08/01 23:01:39 [4851] Number of files: 1 (dir: 1) 2021/08/01 23:01:39 [4851] Number of created files: 0 2021/08/01 23:01:39 [4851] Number of deleted files: 0 2021/08/01 23:01:39 [4851] Number of regular files transferred: 0 2021/08/01 23:01:39 [4851] Total file size: 0 bytes 2021/08/01 23:01:39 [4851] Total transferred file size: 0 bytes 2021/08/01 23:01:39 [4851] Literal data: 0 bytes 2021/08/01 23:01:39 [4851] Matched data: 0 bytes 2021/08/01 23:01:39 [4851] File list size: 0 2021/08/01 23:01:39 [4851] File list generation time: 0.001 seconds 2021/08/01 23:01:39 [4851] File list transfer time: 0.000 seconds 2021/08/01 23:01:39 [4851] Total bytes sent: 52 2021/08/01 23:01:39 [4851] Total bytes received: 110 2021/08/01 23:01:39 [4851] sent 52 bytes received 110 bytes 324.00 bytes/sec 2021/08/01 23:01:39 [4851] total size is 0 speedup is 0.00 2021/08/01 23:01:39 [4851] rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1207) [sender=3.1.3] 2021/08/01 23:02:39 [1273] building file list 2021/08/01 23:02:40 [1273] rsync: chown "/media/storage/202108012301-31863/." failed: Operation not permitted (1) 2021/08/01 23:02:40 [1273] .d...pog... ./ 2021/08/01 23:02:40 [1273] Number of files: 1 (dir: 1) 2021/08/01 23:02:40 [1273] Number of created files: 0 2021/08/01 23:02:40 [1273] Number of deleted files: 0 2021/08/01 23:02:40 [1273] Number of regular files transferred: 0 2021/08/01 23:02:40 [1273] Total file size: 0 bytes 2021/08/01 23:02:40 [1273] Total transferred file size: 0 bytes 2021/08/01 23:02:40 [1273] Literal data: 0 bytes 2021/08/01 23:02:40 [1273] Matched data: 0 bytes 2021/08/01 23:02:40 [1273] File list size: 0 2021/08/01 23:02:40 [1273] File list generation time: 0.001 seconds 2021/08/01 23:02:40 [1273] File list transfer time: 0.000 seconds 2021/08/01 23:02:40 [1273] Total bytes sent: 56 2021/08/01 23:02:40 [1273] Total bytes received: 110 2021/08/01 23:02:40 [1273] sent 56 bytes received 110 bytes 110.67 bytes/sec 2021/08/01 23:02:40 [1273] total size is 0 speedup is 0.00 2021/08/01 23:02:40 [1273] rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1207) [sender=3.1.3] 2021/08/01 23:05:14 [1399] building file list 2021/08/01 23:05:14 [1399] rsync: chown "/media/storage/202108012301-31863/." failed: Operation not permitted (1) 2021/08/01 23:05:14 [1399] .d...pog... ./ 2021/08/01 23:05:14 [1399] Number of files: 1 (dir: 1) 2021/08/01 23:05:14 [1399] Number of created files: 0 2021/08/01 23:05:14 [1399] Number of deleted files: 0 2021/08/01 23:05:14 [1399] Number of regular files transferred: 0 2021/08/01 23:05:14 [1399] Total file size: 0 bytes 2021/08/01 23:05:14 [1399] Total transferred file size: 0 bytes 2021/08/01 23:05:14 [1399] Literal data: 0 bytes 2021/08/01 23:05:14 [1399] Matched data: 0 bytes 2021/08/01 23:05:14 [1399] File list size: 0 2021/08/01 23:05:14 [1399] File list generation time: 0.001 seconds 2021/08/01 23:05:14 [1399] File list transfer time: 0.000 seconds 2021/08/01 23:05:14 [1399] Total bytes sent: 52 2021/08/01 23:05:14 [1399] Total bytes received: 110 2021/08/01 23:05:14 [1399] sent 52 bytes received 110 bytes 324.00 bytes/sec 2021/08/01 23:05:14 [1399] total size is 0 speedup is 0.00 2021/08/01 23:05:14 [1399] rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1207) [sender=3.1.3]

dmpop commented 3 years ago

Really strange. It's the same card you had problems the first time?

ruebyi commented 3 years ago

Nope, another card from another camera

outdoorbits commented 3 years ago

Hi,

could you please send the content of Devices and of Disk space in System info after broken sync?

Thank you, Stefan

Am 7. August 2021 22:04:03 MESZ schrieb ruebyi @.***>:

Nope, another card

-- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/dmpop/little-backup-box/issues/115#issuecomment-894701170 -- Diese Nachricht wurde von meinem Android-Gerät mit K-9 Mail gesendet.

ruebyi commented 3 years ago

Devices NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 232.9G 0 disk └─sda1 8:1 0 232.9G 0 part /media/storage sdb 8:16 1 116.6G 0 disk └─sdb1 8:17 1 116.6G 0 part mmcblk0 179:0 0 119.4G 0 disk ├─mmcblk0p1 179:1 0 256M 0 part /boot └─mmcblk0p2 179:2 0 119.1G 0 part / Disk space Filesystem Size Used Avail Use% Mounted on /dev/root 126G 2.3G 119G 2% / devtmpfs 1.8G 0 1.8G 0% /dev tmpfs 2.1G 0 2.1G 0% /dev/shm tmpfs 2.1G 11M 2.1G 1% /run tmpfs 5.3M 4.1k 5.3M 1% /run/lock tmpfs 2.1G 0 2.1G 0% /sys/fs/cgroup /dev/mmcblk0p1 265M 52M 213M 20% /boot /dev/sda1 251G 112G 139G 45% /media/storage

outdoorbits commented 3 years ago

That means, sdb1 is not mounted, but should be mounted as source, right? Do you use an active USB hub? Maybe it could be a power problem. I have little bit different problems with xqd-cards and I'm working on a more robust mount-strategy. I will tell you ...

Stefan

Am 7. August 2021 22:30:43 MESZ schrieb ruebyi @.***>:

Devices

NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT

sda 8:0 0 232.9G 0 disk > └─sda1 8:1 0 232.9G 0 part /media/storage

sdb 8:16 1 116.6G 0 disk > └─sdb1 8:17 1 116.6G 0 part > mmcblk0 179:0 0 119.4G 0 disk > ├─mmcblk0p1 179:1 0 256M 0 part /boot

└─mmcblk0p2 179:2 0 119.1G 0 part /

Disk space

Filesystem Size Used Avail Use% Mounted on

/dev/root 126G 2.3G 119G 2% /

devtmpfs 1.8G 0 1.8G 0% /dev

tmpfs 2.1G 0 2.1G 0% /dev/shm

tmpfs 2.1G 11M 2.1G 1% /run

tmpfs 5.3M 4.1k 5.3M 1% /run/lock

tmpfs 2.1G 0 2.1G 0% /sys/fs/cgroup

/dev/mmcblk0p1 265M 52M 213M 20% /boot

/dev/sda1 251G 112G 139G 45% /media/storage

-- > You are receiving this because you commented.

Reply to this email directly or view it on GitHub:

https://github.com/dmpop/little-backup-box/issues/115#issuecomment-894703820 -- Diese Nachricht wurde von meinem Android-Gerät mit K-9 Mail gesendet.

ruebyi commented 3 years ago

Sdb1 is a Gibot Cardreader. it worked already with that card…

I‘m not using a hub, it‘s plugged into the USB 3 Ports of my pi 4 4gb

could it be that it got uncounted after the backup try?

outdoorbits commented 3 years ago

But it isn't mounted. Could you check it with an active hub?

Am 7. August 2021 23:01:48 MESZ schrieb ruebyi @.***>:

Sdb1 is a Gibot Cardreader. > it worked already with that card…

-- > You are receiving this because you commented.

Reply to this email directly or view it on GitHub:

https://github.com/dmpop/little-backup-box/issues/115#issuecomment-894706478 -- Diese Nachricht wurde von meinem Android-Gerät mit K-9 Mail gesendet.

ruebyi commented 3 years ago

I will test it at home - I‘m travelling back tomorrow.

there I will be able to test another Cardreader as well

ruebyi commented 3 years ago

OK, at home:

Raspi with original Power Supply - Same error - no Card is recognized correctly Raspi with another SD Card Reader - SD Card Reader is not Compatipble (Transcend UHSII Reader) Raspi with Active Hub - Same Error - no Card is recognized Correctly Raspi with Active Hub - SD Card Reader and external SSD connected to Hub - No Card is recognized correctly

Even Cards, that already worked are not used correctly anymore.

after all this attempts WebUI was not responisive anymore.

EDIT: if Possible: I'm waiting for the merge of the Pullrequests into Master - then I would like to try a fresh installation.

ruebyi commented 3 years ago

Hi, I just tested the new Build:

Device error!

Type: storage to external Source ID: 202108012223-32238

Backup log: ...

Number of files: 1,835 (reg: 1,826, dir: 9) Number of created files: 1,834 (reg: 1,826, dir: 8) Number of deleted files: 0 Number of regular files transferred: 1,826 Total file size: 29.45G bytes Total transferred file size: 29.45G bytes Literal data: 29.45G bytes Matched data: 0 bytes File list size: 0 File list generation time: 0.001 seconds File list transfer time: 0.000 seconds Total bytes sent: 29.46G Total bytes received: 34.76K

sent 29.46G bytes received 34.76K bytes 29.53M bytes/sec total size is 29.45G speedup is 1.00

I'm not able to see what the error is...

dmpop commented 3 years ago

I had the same error. I reformatted the backup USB storage device, and the error was gone. Please note that both developers are traveling now, so we won't be able to do any troubleshooting in the next couple of weeks.

ruebyi commented 3 years ago

Have a good journey! I will try to format and report back

dmpop commented 3 years ago

I've identified one possible cause of the "Lost device" error. It has something to do with the USB 3 ports of Raspberry Pi. If two devices connected to both USB 3 ports draw too much power, the Raspberry Pi "loses" one of them. It seems to be some sort of hardware limitation, and the only way to prevent this from happening is to use one USB 2 and one USB 3 port. Of course, this means a slower transfer speeds, but there is nothing can be done about it, I'm afraid.

ruebyi commented 3 years ago

Ok, Strange That it worked for some Cards then… Thank you!

outdoorbits commented 3 years ago

What happens, if you try to mount the device in webUI/tools?

thelightdecides commented 3 years ago

I have a similar problem. When using CFexpress cards, the copy process stops after about 1500 files. Either the SSD or the memory card is no longer found. Using an active hub or using a USB 2 and a USB 3 port solves the problem. The problem does not occur with SD cards, even 256GB SD cards are copied without problems.

dmpop commented 3 years ago

I have a suspicion that it might have something to do with overheating. It happened to me a couple of times too when I was transferring 1400+ files. I noticed that the USB stick and the USB ports on Raspberry PI got very, very hot. I guess next time I do a backup, I'll stick my Raspberry Pi into a fridge and see it it helps. 😄

dmpop commented 3 years ago

I've rolled back the code in the main branch to the last stable version. So this bug is no longer valid for the current version of Little Backup Box.

The newest code with all the improvements (and bugs :wink: ) is now in the development branch. I've assigned this bug to Stefan. Hopefully he'll be willing to fix it.

outdoorbits commented 3 years ago

Hi,

of course, but I'm still in holiday. Next week I'll be back and try to bugfix.

Have a good time! Stefan

Am 6. September 2021 09:03:05 MESZ schrieb Dmitri Popov @.***>:

I've rolled back the code in the main branch to the last stable version. So this bug is no longer valid for the current version of Little Backup Box.

The newest code with all the improvements (and bugs :wink: ) is now in the development branch. I've assigned this bug to Stefan. Hopefully he'll be willing to fix it.

-- You are receiving this because you were assigned. Reply to this email directly or view it on GitHub: https://github.com/dmpop/little-backup-box/issues/115#issuecomment-913395475

outdoorbits commented 3 years ago

I see two issues:

"Operation not permitted" could mean the card is corrupted. Please check by the fsck-tool.
The message "Device error!" means, the device is not mounted anymore directly after the backup. Probably this is not a software-issue but a hardware-issue. Could you please check in "System info": Is the device is still mounted? If yes, everything worked fine and there would be a problem about the error-detection-routine that pushes "Device error!". If not, I would expect a hardware-problem. If the device was lost, it would be possible to code a loop witch retries the backup for a maximum of config-defined times. If it's a problem of overheating as Dimitri mentioned, a second run after cooling down could be the solution. In a second run rsync just checks date and size of existing files, so it is much faster and missing files could be backuped.

ruebyi commented 2 years ago

Hi, is there something I can do to support you on this issue?

outdoorbits commented 2 years ago

Hi,

I'm just hunting for a bug, then I would like to contact you for testing again.

Best, Stefan

Am 20. November 2021 11:21:45 MEZ schrieb ruebyi @.***>:

Hi, is there something I can do to support you on this issue?

-- You are receiving this because you were assigned. Reply to this email directly or view it on GitHub: https://github.com/dmpop/little-backup-box/issues/115#issuecomment-974628098

outdoorbits commented 2 years ago

Hi,

again, I did many changes and bugfixes and my last tests have been successfully. All my last contribs are not yet in https://github.com/dmpop/little-backup-box but in my https://github.com/stefans73/little-backup-box . I am very excited to get (hopefully good!) news from you. Many technical details are fundamental different to the main branch now, so installation on a new bullseye-lite is necessary.

Thank you for testing! Stefan

Mit meinem angefügten öffentlichen Schlüssel "0xE61B756E.asc" können Sie gerne verschlüsselte Mails an mich senden. Näheres auch unter https://www.gpg4win.de/

Am 2021-11-20 11:21, schrieb ruebyi:

Hi, is there something I can do to support you on this issue?

-- You are receiving this because you were assigned. Reply to this email directly, view it on GitHub [1], or unsubscribe [2].

Links:

[1] https://github.com/dmpop/little-backup-box/issues/115#issuecomment-974628098 [2] https://github.com/notifications/unsubscribe-auth/AKY6K7LB6DHC6TD5A3366MLUM5ZDTANCNFSM5BJYAS6A

outdoorbits commented 2 years ago

Hi Dima,

last mail got wrong, I wanted to ask ruebyi to test. But I would be very happy to get some feedback from you, if you find the time. I spent a night with problems about filebrowser and ended up with replacing it by tinifilemanager. This is/was my trouble: https://stackoverflow.com/questions/70050961/mount-by-php-script-apache but now it works great!

Thank you, Stefan

Mit meinem angefügten öffentlichen Schlüssel "0xE61B756E.asc" können Sie gerne verschlüsselte Mails an mich senden. Näheres auch unter https://www.gpg4win.de/

Am 2021-11-20 11:21, schrieb ruebyi:

Hi, is there something I can do to support you on this issue?

-- You are receiving this because you were assigned. Reply to this email directly, view it on GitHub [1], or unsubscribe [2].

Links:

[1] https://github.com/dmpop/little-backup-box/issues/115#issuecomment-974628098 [2] https://github.com/notifications/unsubscribe-auth/AKY6K7LB6DHC6TD5A3366MLUM5ZDTANCNFSM5BJYAS6A

ruebyi commented 2 years ago

I will test your build tonight I think!

ruebyi commented 2 years ago

Seems to work great! I like the new Features and the new Filebrowser

outdoorbits commented 2 years ago

Hi,

I'm really relieved to read this. Did you also test the cards causing problems before? Could you please tell me, which camera you use? Did you backup by cardreader or directly from camera? I would like to publish compatible systems.

Thank you, Stefan

ruebyi commented 2 years ago

OK, Yesterday I used 2 UHS II SD Cards from Olympus Cameras - If you want I can test today Micor SDs from DJI and GoPro along with Olympus Cards

I'm using a cheap Cardreader from Gibot.

I will report when I Backkuped the next four to five Cards that I have here with me

outdoorbits commented 2 years ago

Hi,

of course, please let me know about. I will add all this to https://github.com/stefans73/little-backup-box/wiki/Compatibility

Thank you! Stefan

thelightdecides commented 2 years ago

Hi, thank you very much for trying to solve the problem. I installed your version and made a short test. Unfortunately, the backup did not complete. During the first test I got this error message: "rsync: [sender] Write error: broken pipe (32)". After rebooting, I tried again, after a few hundred files: "2021/11/25 22:07:55 [4317] rsync: [sender] send_files could not open "/media/source/fill/BJ6A6757.CR3": Input/Output error (5) ". Devices NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 931.5G 0 disk -sda1 8:1 0 931.5G 0 part /media/storage sdc 8:32 0 119.2G 0 disk -sdc1 8:33 0 119.2G 0 part mmcblk0 179:0 0 29.8G 0 disk |-mmcblk0p1 179:1 0 256M 0 part /boot `-mmcblk0p2 179:2 0 29.6G 0 part / Disk space File system Size Used Available Usage% Mounted in /dev/root 32G 2.3G 28G 8% / devtmpfs 1.9G 0 1.9G 0% /dev tmpfs 2.1G 0 2.1G 0% /dev/shm tmpfs 806M 2.2M 803M 1% /run tmpfs 5.3M 4.1k 5.3M 1% /run/lock /dev/mmcblk0p1 265M 51M 214M 20% /boot /dev/sda1 1.1T 143G 858G 15% /media/storage /dev/sdb1 129G 128G 39M 100% /media/source

I think the Pi has lost the reader again, and it was not remounted correctly. Unfortunately, I don't know much about Linux and can't help finding the problem.

Hardware I used: Samsung T5 SSD, SanDisk Extreme Pro CFexpress 128GB, Delkin Card-Reader.

I will try with another cardreader tomorrow and also with SD cards.

Best regards Thomas

outdoorbits commented 2 years ago

Hi,

for me there are a lot of questions open. I will upload a new version next week with extended logging options. If you would spend some time for testing again and send me that logfile, I hope the problem will get clear.

Thank you, Stefan

ruebyi commented 2 years ago

Today - same Hardware - a second card didn't work. The same Card and the same Reader like two days ago...

outdoorbits commented 2 years ago

A new version is online (https://github.com/stefans73/little-backup-box). I changed a lot about logging and also a few things about restarting the backup in case of an error. Please help me to address the problem and send me your log file. Please have a look at https://github.com/stefans73/little-backup-box/wiki/Debuggin-and-log-file first.

Thank you, Stefan

thelightdecides commented 2 years ago

I again used a CFexpress card and the original Pi power supply. The first backup runs fine, no error. Then I deleted the .id file on the card and tried again. Now the card should be a new card for the backup, but its looks like the LBB just wanted to copy 188 files? And the device is lost. I will try again later.

little-backup-box-128GB-CFexpress-Pi-Netzteil-2-Error.txt

outdoorbits commented 2 years ago

Hi, the log file did not start at backup-start, but I didn't see one successful mount of the storage-device. Did you check/repair this by tools and did you try to mount by tools?

Stefan

thelightdecides commented 2 years ago

No, no repair after the first succsessfull backup. I only deleted the id file on the card. I try again later, and send a new file then.

Von: stefans73 @.> Gesendet: Montag, 29. November 2021 16:47 An: dmpop/little-backup-box @.> Cc: thelightdecides @.>; Comment @.> Betreff: Re: [dmpop/little-backup-box] One SD Card won't backup (#115)

Hi, the log file did not start at backup-start, but I didn't see one successful mount of the storage-device. Did you check/repair this by tools and did you try to mount by tools?

Stefan

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/dmpop/little-backup-box/issues/115#issuecomment-981760907, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AOKQOR33SKHREDIRSX2TLHTUOOOA7ANCNFSM5BJYAS6A. Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

outdoorbits commented 2 years ago

The problems seems to be on the destination-device...

thelightdecides commented 2 years ago

Yes, i connected the SSD to my Windows computer and it shows an error. I found 2587 files has been copied before the pi lost the device.

Holen Sie sich Outlook für Androidhttps://aka.ms/AAb9ysg

From: stefans73 @.> Sent: Monday, November 29, 2021 4:55:00 PM To: dmpop/little-backup-box @.> Cc: thelightdecides @.>; Comment @.> Subject: Re: [dmpop/little-backup-box] One SD Card won't backup (#115)

The problems seems to be on the destination-device...

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/dmpop/little-backup-box/issues/115#issuecomment-981767493, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AOKQOR7JE5LRC4ZMZG4OM4TUOOO5JANCNFSM5BJYAS6A. Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

thelightdecides commented 2 years ago

Here is one more log-file: little-backup-box (3).log

outdoorbits commented 2 years ago

Here both devices got lost (maybe at the same time) and they are even not visible for linux lsblk anymore. They are like hardware-disconnected! Strange.

But this seems to happen after long intensive activity only? Loosing both devices after a long period of high activity - could it be, the controller of the pi is overheated? What if you power down the rpi after a broken backup, wait some time to cool down and retry? (Maybe you have to repair the partitions first, use webUI/Tools.) Only files not synced before are planed to sync and synced in secondary tries. So you maybe can backup step by step.

To get perhaps a little more information about mount-problems, I modified the log a little (catch message of mount-command). If you find the time, could you please:

* install the latest version
* choose Default backup mode: none
* set the log-level to 3 (and activate sync-log)
* delete log
* connect the devices
* Tools: Check and repair both relevant partitions
* Tools: If partitions are mounted: click Remove USB 1/ Remove USB 2
* unplug the source device
* Start backup and connect the source device when asked for
* download the log file ...

Maybe easier: Can you copy all the files in one session from your source to your ssd using your windows-machine? If not, it could be a device-specific problem.

thelightdecides commented 2 years ago

ok, I try do this tomorrow.

Probably not the best solution, but could you automatically stop the backup process after e.g. 1000 files and restart it, maybe with a short pause (maybe the temperature on some components rises too far and then the system becomes unstable). The problems usually occur after 1500 or more files. Then you could probably also prevent the errors on the drives. The problems occur only with the CFexpress cards, SD cards of the same size and also larger are copied without problems. I used a 256GB SD card and the backup was no problem.

outdoorbits commented 2 years ago

Breaking the process by timeout also means loosing information about what's synced. I did anther approach. If a device-lost occurs and the sync-process took longer than an defined time and returned an error-code, it expects overheating and waits some time before next retry for cooling. For now I suspect overheating, if sync took >= 120 sec. In this case there is a break of 30 sec for cooling. Unfortunately I can't test this. Could you tell me, after minimal how many seconds this error occurs? Are 30 sec cooling sufficient? You find the updated version at https://github.com/stefans73/little-backup-box

thelightdecides commented 2 years ago

I haven't had time to test this yet. I hope tomorrow.

I have tried another SSD. SanDisk Extreme portable 2TB in combination with the CFexpress card reader. Then the system becomes even more unstable. With earlier versions of LBB I once used an active hub. When I try that now, the drives are mounted but the copy process either doesn't start at all or it is extremely slow. Have you tried this? Can you confirm that an active hub still works? I believe some devices require more or more stable power to operate. My very cheap USB power meter shows very fluctuating values on the Pi at the SSD and card reader. On the laptop this is much more stable. In the end, the whole thing needs to work later with a powerbank if you want to backup pictures on the go. Maybe an active hub solves all the problems.

outdoorbits commented 2 years ago

I´m using XQD-cards and I had a lot of trouble (only) with them and I guess, you have the same. As you described, I also had unstable power situations. Since about one week I use a powerd USB-hub and everything is working stable and without any errors! Yes, it works with an active hub! I use an Anker USB 3.0 70W device, but of course, that´s not what I need for travelling. If you know an active hub usable with a powerbank, I´m very interested! On the other side, XQD-backup to my Sandisk portable extreme 1TB SSD was more stable in combination with my powerbank than an 5V/3A power supply.

When you find the time, I would be very interested in these times:

backup start to break
break until devices are visible again (in System Info)

thelightdecides commented 2 years ago

Some time ago I used this with a HDD. https://www.amazon.de/CSL-Verteiler-Ultrabook-Tablet-PC-bus-powered/dp/B01K7RR3W8/ref=nav_ya_signin?__mk_de_DE=%C3%85M%C3%85%C5%BD%C3%95%C3%91&keywords=usb+hub+csl&qid=1638484908&sr=8-6& .That had worked well. Instead of the power supply, I connected the hub to my powerbank (https://www.thelightdecides.com/reise-fotografie-blog/little-backup-box) . But at the moment the hub makes problems in combination with the SSD and/or the card reader. I have tried the new program. On two runs it worked with the 30s, the card reader was mounted again and the copy process completed. On one run, both devices were lost and could not be reconnected, even not manually via the mount button. I don't know if it is possible to use less than 30s (I was too late to watch the remount), if it is a temperature problem, the time depends on the ambient temperature.

little-backup-box-fehler-kein-mount-moeglich.txt little-backup-box-source-remount.txt

outdoorbits / little-backup-box

One SD Card won't backup #115

Links:

Links: