Closed kretz closed 7 months ago
Well that's odd. As far as I can tell, nothing special happened on May 2nd or specifically around that where there were any changes made. Is that date a specific date or just something you see that worked in general? Since May 1st, a new Alpine Linux came out so maybe that could be an issue. My timemachine backed up last night just fine here but I know that's one data point.
I could try all old images and find out which ones work. Let me get back on that!
To maybe narrow things down, Alpine 3.18 came out on May 9th so maybe try out the May 8th image (smb-20230508
) vs the May 9th (smb-20230509
) and see if that's where things stopped working. That's the main thing where I could see things maybe going wrong.
I have the same thing, and it is not working after an image revert.
2023-06-21 13:40:59 Cleared pending cancellation request
2023-06-21 13:41:14 com.apple.backupd.sandbox.xpc: connection invalid
2023-06-21 13:41:19 com.apple.backupd.sandbox.xpc: connection invalid
2023-06-21 13:41:32 Messaging backupd to start backup attempt for destination ID A24A8F65-2003-4A78-B462-256DE9164CBA)
2023-06-21 13:41:32 com.apple.backupd.xpc: connection invalid
2023-06-21 13:41:32 Starting manual backup
2023-06-21 13:41:32 Failed to create volume info from disk '<TMDisk: 0x138026a00> '/System/Volumes/Data/home'', error: missingURLForRemounting
2023-06-21 13:41:32 Rejecting candidate mount point: /Volumes/TimeMachine, not owned by root
2023-06-21 13:41:32 Attempting to mount 'smb://timemachine@timemachine._smb._tcp.local/TimeMachine'
2023-06-21 13:41:33 Initial network volume options for 'TimeMachine' {disablePrimaryReconnect: 0, disableSecondaryReconnect: 0, reconnectTimeOut: 0, QoS: 0x0, attributes: 0x1C}
2023-06-21 13:41:33 Configured network volume options for 'TimeMachine' {disablePrimaryReconnect: 0, disableSecondaryReconnect: 0, reconnectTimeOut: 30, QoS: 0x20, attributes: 0x1C}
2023-06-21 13:41:33 Mounted 'smb://timemachine@timemachine._smb._tcp.local/TimeMachine' at '/Volumes/.timemachine/timemachine._smb._tcp.local/A615371B-BCC9-4945-A777-AA029E111643/TimeMachine' (973.88 GB of 2.2 TB available)
2023-06-21 13:41:34 Skipping periodic backup verification: not needed for an APFS sparsebundle
2023-06-21 13:41:34 'mbp-torcsi.sparsebundle' may need resizing - current logical size is 1.04 TB (1,044,536,046,080 bytes), size limit is 2.09 TB (2,089,072,092,774 bytes)
2023-06-21 13:42:32 Failed to get name of volume with mountpoint 'file:///Volumes/.timemachine/timemachine._smb._tcp.local/A615371B-BCC9-4945-A777-AA029E111643/TimeMachine/', error: Error Domain=NSCocoaErrorDomain Code=257 "The file “TimeMachine” couldn’t be opened because you don’t have permission to view it." UserInfo={NSURL=file:///Volumes/.timemachine/timemachine._smb._tcp.local/A615371B-BCC9-4945-A777-AA029E111643/TimeMachine/, NSFilePath=/Volumes/.timemachine/timemachine._smb._tcp.local/A615371B-BCC9-4945-A777-AA029E111643/TimeMachine, NSUnderlyingError=0x600002a19620 {Error Domain=NSPOSIXErrorDomain Code=13 "Permission denied"}}
2023-06-21 13:42:32 Failed to get name of volume with mountpoint 'file:///Volumes/.timemachine/timemachine._smb._tcp.local/A615371B-BCC9-4945-A777-AA029E111643/TimeMachine/', error: Error Domain=NSCocoaErrorDomain Code=257 "The file “TimeMachine” couldn’t be opened because you don’t have permission to view it." UserInfo={NSURL=file:///Volumes/.timemachine/timemachine._smb._tcp.local/A615371B-BCC9-4945-A777-AA029E111643/TimeMachine/, NSFilePath=/Volumes/.timemachine/timemachine._smb._tcp.local/A615371B-BCC9-4945-A777-AA029E111643/TimeMachine, NSUnderlyingError=0x600002a18c00 {Error Domain=NSPOSIXErrorDomain Code=13 "Permission denied"}}
2023-06-21 13:42:32 Failed to create volume info from disk '<TMDisk: 0x126024e00> '/Volumes/.timemachine/timemachine._smb._tcp.local/A615371B-BCC9-4945-A777-AA029E111643/TimeMachine'', error: missingName
2023-06-21 13:42:32 Failed to create volume info from disk '<TMDisk: 0x126023a00> '/System/Volumes/Data/home'', error: missingURLForRemounting
2023-06-21 13:42:51 com.apple.backupd.sandbox.xpc: connection invalid
2023-06-21 13:42:55 com.apple.backupd.sandbox.xpc: connection invalid
It stopped to work for me 10 days ago, and I went with a 2 month old image at than. I did not updated my system in the last 2 months (just today with a restart on my mac, and multiple restarts and image changes on my nas). I have no idea what changed.
Confirming same behavior. Rolling back to timemachine:smb-20230501 worked for me also. MacOs was acting like it didn't have RW access to timecapsule, I couldn't create a folder on the destination from MacOs mounted to timemachine.
I appreciate the additional context from the both of you. Could you share what hardware you have and what OS version you're running?
Happy to help. I'm running it on an Asus Tinkerboard (arm) on Armbian Focal 21.02.3 with Linux Kernel 5.9.14-rockchip64. Docker version 19.03.13, build 4484c46.
Ubuntu 22.04.2 LTS (GNU/Linux 5.15.0-60-generic x86_64)
, Docker version 23.0.3, build 3e7cbfd
The HW is a custom build Asrock J3160-ITX NAS. (The mac is a 2021 14inch pro with M1 Max and currently on 13.4 (22F66))
And to be clear timemachine:smb-20230501
is not working for me. I (re)installed the system at 2023-04-11 so I think I used that version until yesterday, and it stopped working at 2023-06-10 without updating or upgrading anything on my macOs or the NAS. (I started to mess with things bcs I get an OS notification that my backups are older than 10 days.)
My compose looks like this with a macvlan
network driver:
timemachine:
hostname: timemachine
networks:
physical:
ipv4_address: 10.32.45.208
environment:
- CUSTOM_SMB_CONF=false
- CUSTOM_USER=false
- DEBUG_LEVEL=1
- EXTERNAL_CONF=
- HIDE_SHARES=no
- MIMIC_MODEL=TimeCapsule8,119
- TM_USERNAME=timemachine
- TM_GROUPNAME=timemachine
- TM_UID=1000
- TM_GID=1000
- PASSWORD=timemachine
- SET_PERMISSIONS=false
- SHARE_NAME=TimeMachine
- SMB_INHERIT_PERMISSIONS=no
- SMB_NFS_ACES=yes
- SMB_METADATA=stream
- SMB_PORT=445
- SMB_VFS_OBJECTS=acl_xattr fruit streams_xattr
- VOLUME_SIZE_LIMIT=2T
- WORKGROUP=WORKGROUP
restart: unless-stopped
ports:
- "137:137/udp"
- "138:138/udp"
- "139:139"
- "445:445"
volumes:
- /hdd2/timemachine/data:/opt/timemachine
- /hdd2/timemachine/samba:/var/lib/samba
- /hdd2/timemachine/cache-samba:/var/cache/samba
- /hdd2/timemachine/run-samba:/run/samba
ulimits:
nofile:
soft: 65536
hard: 65536
container_name: timemachine
image: mbentley/timemachine:smb-20230501
container logs;
INFO: CUSTOM_SMB_CONF=false; generating [global] section of /etc/samba/smb.conf...
INFO: Creating /var/log/samba/cores
INFO: Avahi - generating base configuration in /etc/avahi/services/smbd.service...
INFO: Avahi - adding the 'dk0', 'TimeMachine' share txt-record to /etc/avahi/services/smbd.service...
INFO: Group timemachine doesn't exist; creating...
INFO: User timemachine doesn't exist; creating...
INFO: Using default password: timemachine
chpasswd: password for 'timemachine' changed
INFO: INFO: CUSTOM_SMB_CONF=false; generating [TimeMachine] section of /etc/samba/smb.conf...
INFO: Samba - Created User timemachine password set to none.
INFO: Samba - Enabled user timemachine.
INFO: Samba - setting password
INFO: SET_PERMISSIONS=false; not setting ownership and permissions for /opt/timemachine
INFO: Avahi - completing the configuration in /etc/avahi/services/smbd.service...
INFO: running test for xattr support on your time machine persistent storage location...
INFO: xattr test successful - your persistent data store supports xattrs
INFO: entrypoint complete; executing 's6-svscan /etc/s6'
dbus socket not yet available; sleeping...
nmbd version 4.16.10 started.
Copyright Andrew Tridgell and the Samba Team 1992-2022
smbd version 4.16.10 started.
Copyright Andrew Tridgell and the Samba Team 1992-2022
INFO: Profiling support unavailable in this build.
Found user 'avahi' (UID 86) and group 'avahi' (GID 86).
Successfully dropped root privileges.
avahi-daemon 0.8 starting up.
WARNING: No NSS support for mDNS detected, consider installing nss-mdns!
Loading service file /etc/avahi/services/smbd.service.
Joining mDNS multicast group on interface eth0.IPv4 with address 10.32.45.208.
New relevant interface eth0.IPv4 for mDNS.
Joining mDNS multicast group on interface lo.IPv4 with address 127.0.0.1.
New relevant interface lo.IPv4 for mDNS.
Network interface enumeration completed.
Registering new address record for 10.32.45.208 on eth0.IPv4.
Registering new address record for 127.0.0.1 on lo.IPv4.
Server startup complete. Host name is timemachine.local. Local service cookie is 614197164.
ndr_pull_xattr_DosInfo: ndr_pull_error(Bad Switch): Bad switch value 5 at librpc/gen_ndr/ndr_xattr.c:390 at librpc/gen_ndr/ndr_xattr.c:390
parse_dos_attribute_blob: bad ndr decode from EA on file mbp-torcsi.sparsebundle: Error = Bad Switch
ndr_pull_xattr_DosInfo: ndr_pull_error(Bad Switch): Bad switch value 5 at librpc/gen_ndr/ndr_xattr.c:390 at librpc/gen_ndr/ndr_xattr.c:390
parse_dos_attribute_blob: bad ndr decode from EA on file mbp-torcsi.sparsebundle: Error = Bad Switch
ndr_pull_xattr_DosInfo: ndr_pull_error(Bad Switch): Bad switch value 5 at librpc/gen_ndr/ndr_xattr.c:390 at librpc/gen_ndr/ndr_xattr.c:390
parse_dos_attribute_blob: bad ndr decode from EA on file mbp-torcsi.sparsebundle: Error = Bad Switch
The last lines are in repeat for pages.
Happy to help. I'm running it on an Asus Tinkerboard (arm) on Armbian Focal 21.02.3 with Linux Kernel 5.9.14-rockchip64. Docker version 19.03.13, build 4484c46.
Sorry, I should have been more specific - curious about your Mac.
No worries! I have 4 macs backing up to the timecapsule container. None of them would work. The oldest is a 2013 MBP 13" and the newest is a 2021 M1 Pro 14" running Ventura. They now work after the roll back.
On Thu, Jun 22, 2023 at 7:43 AM Matt Bentley @.***> wrote:
Happy to help. I'm running it on an Asus Tinkerboard (arm) on Armbian Focal 21.02.3 with Linux Kernel 5.9.14-rockchip64. Docker version 19.03.13, build 4484c46.
Sorry, I should have been more specific - curious about your Mac.
— Reply to this email directly, view it on GitHub https://github.com/mbentley/docker-timemachine/issues/133#issuecomment-1602492914, or unsubscribe https://github.com/notifications/unsubscribe-auth/AH25EVVOV4OUAO3O5YKXEDTXMQVWNANCNFSM6AAAAAAYGQZ3ZM . You are receiving this because you commented.Message ID: @.***>
Well good news - I can reproduce this in my own environment. The bad news is that I didn't have to do anything to reproduce it so I don't know what's triggered it but at least I can do some digging.
*edit: Well, annoying thing is that I just restarted my time machine container and tried to start a backup and it started working so that's not exactly helpful.
Any progress with this? Or this is just in a "not reproduceable" state?
Sorry, no further progress but that is mostly due to not being able to consistently reproduce the problem seeing as just restarting the container seemed to put things back into a working state.
I am running this on a k3s cluster of RPi4 with external drivers and having this issue as well. Tried a lot of images and all then are with the problem. Also, I am backing up a MacBook Pro m1. And I see it across different macOS versions.
Would you point me in the direction of debugging this? I see a lot of permission denied and a connection error. Time Machine is able to finish the backup while I am logged in, but always fail while logged out
Could you share some of the logs? At this point, the only thing that might help is some more specific errors from the logs to search on.
*edit: my time machine backups were failing on my m1 laptop since August 1st as well - no real useful errors, tried to manually start a backup and it kept failing saying it couldn't connect to the share. Restarted the container and it just started working when i manually kicked off a backup this morning. So frustrating.
I just get this message over and over, for all the 3 TM backups.
this error does not make sense, it is about the mounted sparsebundle volume, which is tm's control.
On docker side i do not get errors...
2023-08-13 21:37:16.932364-0300 localhost TimeMachineSettings[17814]: (TimeMachine) [com.apple.TimeMachine:General] Failed to get resource value 'NSURLVolumeURLForRemountingKey' for '/Volumes/.timemachine/192.168.5.3/6FB466CF-9ACC-42CC-8716-4A141671C7B7/TimeMachine03', error: Error Domain=NSCocoaErrorDomain Code=257 "The file “TimeMachine03” couldn’t be opened because you don’t have permission to view it." UserInfo={NSURL=file:///Volumes/.timemachine/192.168.5.3/6FB466CF-9ACC-42CC-8716-4A141671C7B7/TimeMachine03/, NSFilePath=/Volumes/.timemachine/192.168.5.3/6FB466CF-9ACC-42CC-8716-4A141671C7B7/TimeMachine03, NSUnderlyingError=0x600001e7c780 {Error Domain=NSPOSIXErrorDomain Code=13 "Permission denied"}}
is there a user id and group id that i should use? i believe the time machine backupd is not able to access the time machine
The default UID:GID is 1000:1000
for the timemachine user if the user doesn't specify anything. Not sure if mounting the time machine share directly on the host through any other user (if one is present/enabled) would cause any issues or not. It seems like it can mount the sparsebundle but the permission issue is happening when it is attempting to do something with the sparsebundle.
I manually mounted my timemachine shared on my host using Finder by opening it as a smb share (something like smb://192.168.2.162
where 192.168.2.162
is your time machine host), I entered my timemachine user and password when prompted and then I could see the sparsebundles that are stored like this:
I could double click on one (it will probably take a while for it to open) and then from the Terminal, I could see the mounted sparsebundle:
$ df -h | grep com.apple.TimeMachine
com.apple.TimeMachine.2021-03-10-002122.backup@/dev/disk3s1 1.9Ti 370Gi 784Gi 33% 1.6M 8.2G 0% /Volumes/.timemachine/D05AD108-2BA1-435F-91C2-2CA4D52719CE/2021-03-10-002122.backup
At least for me, I could list the directory contents:
$ ls -la /Volumes/.timemachine/D05AD108-2BA1-435F-91C2-2CA4D52719CE/2021-03-10-002122.backup
total 24
drwxr-xr-x@ 5 root wheel 160 Mar 10 2021 ./
drwxr-xr-x 3 root wheel 96 Oct 9 14:41 ../
drwx------ 4 mbentley staff 128 Dec 24 2020 .Spotlight-V100/
drwxr-xr-x@ 6 root wheel 192 Mar 10 2021 2021-03-10-002122.backup/
-rw-r--r--@ 1 root wheel 10782 Mar 9 2021 backup_manifest.plist
No idea if this will help narrow down anything or not but at least I could access my sparsebundle directly to verify that it could see it. I could even drill down into the /Users
folder from that point in time by traversing the folder paths:
$ ls -la /Volumes/.timemachine/D05AD108-2BA1-435F-91C2-2CA4D52719CE/2021-03-10-002122.backup/2021-03-10-002122.backup/Macintosh\ HD\ -\ Data/Users/
total 0
drwxr-xr-x@ 5 root admin 160 Jan 1 2020 ./
drwxr-xr-x@ 23 root wheel 736 Mar 10 2021 ../
-rw-r--r--+ 1 root wheel 0 Jan 1 2020 .localized
drwxrwxrwt@ 4 root wheel 128 Jan 1 2020 Shared/
drwxr-xr-x@ 73 administrator staff 2336 Mar 8 2021 mbentley/
I don't know if this is directly related but I've been having an issue that anytime I seem to upgrade mbentley/docker-timemachine (or maybe even restart the container... need to look into his further), Time Machine stops working. The backup simply fails even though I can see the .sparsebundle backup is still present. It's been going on for about year now and my solution has just been to delete the backup and setup Time Machine again, which obviously isn't ideal.
I did notice when I'm having this issue, if I do df -h | grep com.apple.TimeMachine
, it returns nothing.
If I ls -la
in /Volumes/.timemachine/172.16.16.1/82CF8785-DA9C-4D27-AE4C-8E72D9B23A95
, the directory appears to be empty.
If I delete my backup, setup Time Machine again and create a new backup, then do df -h | grep com.apple.TimeMachine
, I'm now seeing:
com.apple.TimeMachine.2023-12-10-102752.local@/dev/disk3s5 460Gi 111Gi 334Gi 25% 836k 3.5G 0% /Volumes/com.apple.TimeMachine.localsnapshots/Backups.backupdb/My Mac mini/2023-12-10-102752/Data
If I ls -la
in /Volumes/.timemachine/172.16.16.1/C7A506EA-3742-479E-8DD7-F303A5B9418F
, I do see a Backup
directory.
My docker-compose:
samba:
image: mbentley/timemachine:smb
container_name: samba
restart: unless-stopped
environment:
- PASSWORD_FILE=/run/secrets/samba_password
- SHARE_NAME=Backup
secrets:
- samba_password
volumes:
- /home/my_user/backup/timemachine:/opt/timemachine
- timemachine-var-lib-samba:/var/lib/samba
- timemachine-var-cache-samba:/var/cache/samba
- timemachine-run-samba:/run/samba
ports:
- 172.16.16.1:137:137/udp
- 172.16.16.1:138:138/udp
- 172.16.16.1:139:139
- 172.16.16.1:445:445
Update: I restarted the docker container and Time Machine still works. I also restarted my computer and Time Machine still works. So I believe it stops working whenever I upgrade to a new version of the docker container.
It's possible that this could be related to #149. Maybe there are cached files that are conflicting with the setup. I just created a mbentley/timemachine:volume-test
image that does not have any volumes defined in the Dockerfile. If you want to try it, here is an updated compose file based on yours from above:
samba:
image: mbentley/timemachine:volume-test
container_name: samba
restart: unless-stopped
environment:
- PASSWORD_FILE=/run/secrets/samba_password
- SHARE_NAME=Backup
secrets:
- samba_password
volumes:
- /home/my_user/backup/timemachine:/opt/timemachine
tmpfs:
- /run/samba
ports:
- 172.16.16.1:137:137/udp
- 172.16.16.1:138:138/udp
- 172.16.16.1:139:139
- 172.16.16.1:445:445
Just updated my docker compose file and rebuilt. I'm getting a "Time Machine couldn’t verify your backups." when trying to run another backup. I did update to Debian 12.4 yesterday and rebooted my host machine. I went ahead and deleted my backup .sparsebundle and setup Time Machine again. I'll keep an eye on it from here. Thanks for the tip!
One of my favorite errors. Really kills me when it does that, knowing it's going to have to create a new backup, which always seems to take forever for me.
Just updated to the latest version and unfortunately I'm still having the same "Backup Failed" issue with the new docker compose setup. df -h | grep com.apple.TimeMachine
is not returning anything. Up until the update, everything was working fine but that's usually how it's been where any updates seem to break Time Machine for me.
Edit: I'm an idiot, I never updated to the mbentley/timemachine:volume-test
image. I just updated my docker compose and rebuilt the container. I'll keep an eye on it from here.
I deleted my old backups, re-setup Time Machine with the updated docker compose and performed a new backup. Afterwards, I did docker compose up --force-recreate -d samba
to force the issue and unfortunately it's still occurring - backup failed.
I just merged a PR that seems to have helped my backups: https://github.com/mbentley/docker-timemachine/pull/156
If you don’t have SMB_VFS_OBJECTS defined as an env var, just pulling the newest image from today and recreating the containers help.
I just updated to the latest image and Time Machine seems to be working again! I did not delete and re-setup Time Machine either. I'll continue to keep an eye on it and report back but I'm hopeful this fixed the issue for me. Thanks!
Awesome, great to hear! Huge should out to @Alex1s who suggested the change.
Closing for now; feel free to reply if you have further issues.
Describe the Bug
I have been running this image for about a year. I have auto-upgrade enabled of
smb
and some time during May it stopped working.Logging in OSX with
log stream --level debug --predicate 'subsystem == "com.apple.TimeMachine"'
gives the following:Reverting back to tag
mbentley/timemachine:smb-20230501
made things work again.Expected Behavior
Normal working time machine backups over SMB.
Steps to Reproduce
smb
and the problem shows up.smb-20230501
and it is gone.How You're Launching the Container
Additional Context
No response