borgbase / vorta

Desktop Backup Client for Borg Backup
https://vorta.borgbase.com
GNU General Public License v3.0
1.98k stars 132 forks source link

[Flatpak + KWallet] Unable to connect / add repo on borgbase #1753

Open andreaippo opened 1 year ago

andreaippo commented 1 year ago

Description

My Vorta backups stopped around the 8th of July. I'm using the Flatpak app and my repo is hosted by borgbase.com

Suddenly all my scheduled backups started to fail with error code 2.

I tried to remove and re-add the repo to Vorta, and I always get the same error as per screenshot attached. Log also attached.

Spoiler: it seems to be a problem accessing KWallet to retrieve my borg passphrase. I have even tried enabling all permissions for this package using Flatseal, but still I get the same error.

I have ended up installing the packaged version from my distro, and that one works fine.

Reproduction

OS

openSuse Tumbleweed

Version of Vorta

0.8.12

What did you install Vorta with?

Flatpak

Version of Borg

1.2.4

Logs

2023-07-14 15:49:43,919 - vorta.i18n - DEBUG - Loading translation succeeded for ['en', 'en-US', 'en-Latn-US'].
2023-07-14 15:49:43,921 - vorta.scheduler - WARNING - Failed to connect to DBUS interface to detect sleep/resume events
2023-07-14 15:49:44,128 - root - INFO - Using NetworkManagerMonitor NetworkStatusMonitor implementation.
2023-07-14 15:49:44,136 - vorta.network_status.network_manager - WARNING - Couldn't load settings for /org/freedesktop/NetworkManager/Settings/5
Traceback (most recent call last):
  File "/app/lib/python3.10/site-packages/vorta/network_status/network_manager.py", line 53, in get_known_wifis
    settings = self._nm.get_settings(connection_path)
  File "/app/lib/python3.10/site-packages/vorta/network_status/network_manager.py", line 143, in get_settings
    return get_result(settings.call('GetSettings'))
  File "/app/lib/python3.10/site-packages/vorta/network_status/network_manager.py", line 167, in get_result
    raise DBusException("DBus call failed: {}".format(msg.arguments()))
vorta.network_status.network_manager.DBusException: DBus call failed: ['uid 1000 has no permission to perform this operation']
2023-07-14 15:49:44,137 - vorta.network_status.network_manager - WARNING - Couldn't load settings for /org/freedesktop/NetworkManager/Settings/6
Traceback (most recent call last):
  File "/app/lib/python3.10/site-packages/vorta/network_status/network_manager.py", line 53, in get_known_wifis
    settings = self._nm.get_settings(connection_path)
  File "/app/lib/python3.10/site-packages/vorta/network_status/network_manager.py", line 143, in get_settings
    return get_result(settings.call('GetSettings'))
  File "/app/lib/python3.10/site-packages/vorta/network_status/network_manager.py", line 167, in get_result
    raise DBusException("DBus call failed: {}".format(msg.arguments()))
vorta.network_status.network_manager.DBusException: DBus call failed: ['uid 1000 has no permission to perform this operation']
2023-07-14 15:49:44,208 - vorta.borg.jobs_manager - DEBUG - Add job for site default
2023-07-14 15:49:44,209 - vorta.borg.jobs_manager - DEBUG - Start job on site: default
2023-07-14 15:49:44,212 - vorta.borg.borg_job - INFO - Running command /app/bin/borg --version
2023-07-14 15:49:44,686 - vorta.borg.jobs_manager - DEBUG - Finish job for site: default
2023-07-14 15:49:44,686 - vorta.borg.jobs_manager - DEBUG - No more jobs for site: default
2023-07-14 15:49:44,687 - vorta.scheduler - DEBUG - Refreshing all scheduler timers
2023-07-14 15:49:44,692 - vorta.scheduler - DEBUG - Nothing scheduled for profile 1 because of unset repo.
2023-07-14 15:49:50,975 - vorta.keyring.abc - DEBUG - Only available on macOS
2023-07-14 15:49:51,040 - vorta.keyring.abc - DEBUG - Using VortaKWallet5Keyring
2023-07-14 15:50:04,795 - vorta.keyring.abc - DEBUG - Only available on macOS
2023-07-14 15:50:04,797 - vorta.keyring.abc - DEBUG - Using VortaKWallet5Keyring
2023-07-14 15:50:10,124 - vorta.keyring.abc - DEBUG - Only available on macOS
2023-07-14 15:50:10,126 - vorta.keyring.abc - DEBUG - Using VortaKWallet5Keyring
2023-07-14 15:50:10,126 - vorta.borg.borg_job - DEBUG - Using VortaKWallet5Keyring keyring to store passwords.
2023-07-14 15:50:10,128 - vorta.borg.borg_job - DEBUG - Password not found in primary keyring. Falling back to VortaDBKeyring.
2023-07-14 15:50:10,135 - vorta.borg.borg_job - INFO - Running command /app/bin/borg info --info --json --log-json ssh://p6su75vy@p6su75vy.repo.borgbase.com/./repo
2023-07-14 15:50:11,341 - vorta.borg.borg_job - WARNING - Remote: p6su75vy@p6su75vy.repo.borgbase.com: Permission denied (publickey,gssapi-keyex,gssapi-with-mic).
2023-07-14 15:50:11,344 - vorta.borg.borg_job - ERROR - Connection closed by remote host. Is borg working on the server?
2023-07-14 15:50:27,755 - vorta.keyring.abc - DEBUG - Only available on macOS
2023-07-14 15:50:27,758 - vorta.keyring.abc - DEBUG - Using VortaKWallet5Keyring
2023-07-14 15:50:27,758 - vorta.borg.borg_job - DEBUG - Using VortaKWallet5Keyring keyring to store passwords.
2023-07-14 15:50:27,763 - vorta.borg.borg_job - DEBUG - Password not found in primary keyring. Falling back to VortaDBKeyring.
2023-07-14 15:50:27,773 - vorta.borg.borg_job - INFO - Running command /app/bin/borg info --info --json --log-json ssh://p6su75vy@p6su75vy.repo.borgbase.com/./repo
2023-07-14 15:50:28,928 - vorta.borg.borg_job - WARNING - Remote: p6su75vy@p6su75vy.repo.borgbase.com: Permission denied (publickey,gssapi-keyex,gssapi-with-mic).
2023-07-14 15:50:28,932 - vorta.borg.borg_job - ERROR - Connection closed by remote host. Is borg working on the server?
2023-07-14 15:50:32,965 - vorta.keyring.abc - DEBUG - Only available on macOS
2023-07-14 15:50:32,968 - vorta.keyring.abc - DEBUG - Using VortaKWallet5Keyring
2023-07-14 15:50:32,968 - vorta.borg.borg_job - DEBUG - Using VortaKWallet5Keyring keyring to store passwords.
2023-07-14 15:50:32,970 - vorta.borg.borg_job - DEBUG - Password not found in primary keyring. Falling back to VortaDBKeyring.
2023-07-14 15:50:32,977 - vorta.borg.borg_job - INFO - Running command /app/bin/borg info --info --json --log-json ssh://p6su75vy@p6su75vy.repo.borgbase.com/./repo
2023-07-14 15:50:34,288 - vorta.borg.borg_job - WARNING - Remote: p6su75vy@p6su75vy.repo.borgbase.com: Permission denied (publickey,gssapi-keyex,gssapi-with-mic).
2023-07-14 15:50:34,292 - vorta.borg.borg_job - ERROR - Connection closed by remote host. Is borg working on the server?
2023-07-14 15:50:34,683 - vorta.keyring.abc - DEBUG - Only available on macOS
2023-07-14 15:50:34,685 - vorta.keyring.abc - DEBUG - Using VortaKWallet5Keyring
2023-07-14 15:50:34,686 - vorta.borg.borg_job - DEBUG - Using VortaKWallet5Keyring keyring to store passwords.
2023-07-14 15:50:34,690 - vorta.borg.borg_job - DEBUG - Password not found in primary keyring. Falling back to VortaDBKeyring.
2023-07-14 15:50:34,701 - vorta.borg.borg_job - INFO - Running command /app/bin/borg info --info --json --log-json ssh://p6su75vy@p6su75vy.repo.borgbase.com/./repo
2023-07-14 15:50:36,129 - vorta.borg.borg_job - WARNING - Remote: p6su75vy@p6su75vy.repo.borgbase.com: Permission denied (publickey,gssapi-keyex,gssapi-with-mic).
2023-07-14 15:50:36,133 - vorta.borg.borg_job - ERROR - Connection closed by remote host. Is borg working on the server?
2023-07-14 15:54:07,985 - vorta.borg.borg_job - DEBUG - Cancel job on site default
2023-07-14 15:54:07,985 - vorta.borg.jobs_manager - INFO - Finished cancelling all jobs
andreaippo commented 1 year ago

image

And this is what kwallet looks like:

image

andreaippo commented 1 year ago

Ok I have tried to disable the option to use the system wallet to retrieve the repo password, and let Vorta store it somewhere in plaintext.

Now it still fails but one step further:

2023-07-14 16:08:02,163 - vorta.i18n - DEBUG - Loading translation succeeded for ['en', 'en-US', 'en-Latn-US'].
2023-07-14 16:08:02,404 - root - INFO - Using NetworkManagerMonitor NetworkStatusMonitor implementation.
2023-07-14 16:08:02,410 - vorta.network_status.network_manager - WARNING - Couldn't load settings for /org/freedesktop/NetworkManager/Settings/5
Traceback (most recent call last):
  File "/app/lib/python3.10/site-packages/vorta/network_status/network_manager.py", line 53, in get_known_wifis
    settings = self._nm.get_settings(connection_path)
  File "/app/lib/python3.10/site-packages/vorta/network_status/network_manager.py", line 143, in get_settings
    return get_result(settings.call('GetSettings'))
  File "/app/lib/python3.10/site-packages/vorta/network_status/network_manager.py", line 167, in get_result
    raise DBusException("DBus call failed: {}".format(msg.arguments()))
vorta.network_status.network_manager.DBusException: DBus call failed: ['uid 1000 has no permission to perform this operation']
2023-07-14 16:08:02,411 - vorta.network_status.network_manager - WARNING - Couldn't load settings for /org/freedesktop/NetworkManager/Settings/6
Traceback (most recent call last):
  File "/app/lib/python3.10/site-packages/vorta/network_status/network_manager.py", line 53, in get_known_wifis
    settings = self._nm.get_settings(connection_path)
  File "/app/lib/python3.10/site-packages/vorta/network_status/network_manager.py", line 143, in get_settings
    return get_result(settings.call('GetSettings'))
  File "/app/lib/python3.10/site-packages/vorta/network_status/network_manager.py", line 167, in get_result
    raise DBusException("DBus call failed: {}".format(msg.arguments()))
vorta.network_status.network_manager.DBusException: DBus call failed: ['uid 1000 has no permission to perform this operation']
2023-07-14 16:08:02,488 - vorta.borg.jobs_manager - DEBUG - Add job for site default
2023-07-14 16:08:02,489 - vorta.borg.jobs_manager - DEBUG - Start job on site: default
2023-07-14 16:08:02,496 - vorta.borg.borg_job - INFO - Running command /app/bin/borg --version
2023-07-14 16:08:02,900 - vorta.borg.jobs_manager - DEBUG - Finish job for site: default
2023-07-14 16:08:02,901 - vorta.borg.jobs_manager - DEBUG - No more jobs for site: default
2023-07-14 16:08:02,901 - vorta.scheduler - DEBUG - Refreshing all scheduler timers
2023-07-14 16:08:02,906 - vorta.scheduler - DEBUG - Nothing scheduled for profile 1 because of unset repo.
2023-07-14 16:08:06,092 - vorta.keyring.abc - DEBUG - Only available on macOS
2023-07-14 16:08:06,153 - vorta.keyring.abc - DEBUG - Using VortaKWallet5Keyring
2023-07-14 16:08:09,801 - vorta.keyring.abc - DEBUG - Only available on macOS
2023-07-14 16:08:09,803 - vorta.keyring.abc - DEBUG - Using VortaKWallet5Keyring
2023-07-14 16:08:09,807 - vorta.keyring.kwallet - DEBUG - Retrieved password for repo ssh://p6su75vy@p6su75vy.repo.borgbase.com/./repo
2023-07-14 16:08:15,339 - vorta.keyring.abc - DEBUG - Only available on macOS
2023-07-14 16:08:15,341 - vorta.keyring.abc - DEBUG - Using VortaKWallet5Keyring
2023-07-14 16:08:15,341 - vorta.borg.borg_job - DEBUG - Using VortaKWallet5Keyring keyring to store passwords.
2023-07-14 16:08:15,344 - vorta.keyring.kwallet - DEBUG - Retrieved password for repo ssh://p6su75vy@p6su75vy.repo.borgbase.com/./repo
2023-07-14 16:08:15,349 - vorta.borg.borg_job - INFO - Running command /app/bin/borg info --info --json --log-json ssh://p6su75vy@p6su75vy.repo.borgbase.com/./repo
2023-07-14 16:08:16,562 - vorta.borg.borg_job - WARNING - Remote: p6su75vy@p6su75vy.repo.borgbase.com: Permission denied (publickey,gssapi-keyex,gssapi-with-mic).
2023-07-14 16:08:16,566 - vorta.borg.borg_job - ERROR - Connection closed by remote host. Is borg working on the server?
2023-07-14 16:08:23,932 - vorta.keyring.abc - DEBUG - Only available on macOS
2023-07-14 16:08:23,932 - vorta.keyring.abc - DEBUG - Using VortaDBKeyring
2023-07-14 16:08:25,431 - vorta.keyring.abc - DEBUG - Only available on macOS
2023-07-14 16:08:25,431 - vorta.keyring.abc - DEBUG - Using VortaDBKeyring
2023-07-14 16:08:29,475 - vorta.keyring.abc - DEBUG - Only available on macOS
2023-07-14 16:08:29,475 - vorta.keyring.abc - DEBUG - Using VortaDBKeyring
2023-07-14 16:08:29,475 - vorta.borg.borg_job - DEBUG - Using VortaDBKeyring keyring to store passwords.
2023-07-14 16:08:29,484 - vorta.borg.borg_job - INFO - Running command /app/bin/borg info --info --json --log-json ssh://p6su75vy@p6su75vy.repo.borgbase.com/./repo
2023-07-14 16:08:30,716 - vorta.borg.borg_job - WARNING - Remote: p6su75vy@p6su75vy.repo.borgbase.com: Permission denied (publickey,gssapi-keyex,gssapi-with-mic).
2023-07-14 16:08:30,720 - vorta.borg.borg_job - ERROR - Connection closed by remote host. Is borg working on the server?
2023-07-14 16:08:36,358 - vorta.keyring.abc - DEBUG - Only available on macOS
2023-07-14 16:08:36,359 - vorta.keyring.abc - DEBUG - Using VortaDBKeyring
2023-07-14 16:08:36,359 - vorta.borg.borg_job - DEBUG - Using VortaDBKeyring keyring to store passwords.
2023-07-14 16:08:36,366 - vorta.borg.borg_job - INFO - Running command /app/bin/borg info --info --json --log-json ssh://p6su75vy@p6su75vy.repo.borgbase.com/./repo
2023-07-14 16:08:37,572 - vorta.borg.borg_job - WARNING - Remote: p6su75vy@p6su75vy.repo.borgbase.com: Permission denied (publickey,gssapi-keyex,gssapi-with-mic).
2023-07-14 16:08:37,575 - vorta.borg.borg_job - ERROR - Connection closed by remote host. Is borg working on the server?
m3nu commented 1 year ago

This sounds more like a support request and you should write to hello @ borgbase.com

Upon checking, I don't see login errors for your user. Last login was 1h ago.

Since Flatpak is sandboxed, it may not use the correct key right away? We have an FAQ on it here.

Jul 14 14:20:46 box-eu28.borgbase.com sshd[381142]: Accepted publickey for p6su75vy from 93.67.*** port 47226 ssh2: ED25519 SHA256:SdHD/Ya9/QIWTUjb8f0u***
Jul 14 14:20:46 box-eu28.borgbase.com sshd[381142]: pam_unix(sshd:session): session opened for user p6su75vy(uid=1947) by (uid=0)
real-yfprojects commented 1 year ago

Yeah I would guess that this is an issue with permissions of an ssh key.

andreaippo commented 5 months ago

Removed to keep things short. Simple scenario to reproduce the issue can be found in the next comment.

andreaippo commented 5 months ago

Ok I will try to provide a minimal scenario to reproduce the issue. @Hofer-Julian @Parnassius FYI :)

Pre-requisites

Make sure you have the following in your ~/.profile. Please adapt to your system's path to the ksshaskpass executable:

# For SSH agent systemd user unit
export SSH_AUTH_SOCK="$XDG_RUNTIME_DIR/ssh-agent.socket"
# To force SSH agent to prompt passphrases via ksshaskpass, which allows to store them inside the KWallet
export SSH_ASKPASS='/usr/libexec/ssh/ksshaskpass'
export SSH_ASKPASS_REQUIRE=prefer

Create a systemd user service as follows:

# Create directory for systemd user units
mkdir -p ~/.config/systemd/user

echo '[Unit]
Description=SSH key agent

[Service]
Type=simple
Environment=SSH_AUTH_SOCK=%t/ssh-agent.socket
ExecStart=/usr/bin/ssh-agent -D -a $SSH_AUTH_SOCK

[Install]
WantedBy=default.target' >> ~/.config/systemd/user/ssh-agent.service

systemctl --user enable --now ssh-agent.service

This stuff is needed to make sure the ssh-agent is run at login and that the interaction with the KDE Wallet Manager is properly setup, so that when an ssh private key passphrase is requested, the user is prompted with a UI (ksshaskpass) and can store the passphrase in the KWalletManager just by enabling the Remember checkbox.

Finally, make a backup of your ~/.ssh/config and use one with only the contents below instead. It includes the dummy borgbase repo (1GB) I created for this purpose:

Host v28lu6o4.repo.borgbase.com
    IdentityFile ~/.ssh/test-vorta-issue
    AddKeysToAgent  yes

Save this dummy key pair in your ~/.ssh. It's been set as the one authorized for the dummy borgbase repo:

private key, name it "test-vorta-issue" (chmod 600):

-----BEGIN OPENSSH PRIVATE KEY-----
b3BlbnNzaC1rZXktdjEAAAAACmFlczI1Ni1jdHIAAAAGYmNyeXB0AAAAGAAAABBD1ZXH/p
tTQtUq7fzFegdpAAAAGAAAAAEAAAAzAAAAC3NzaC1lZDI1NTE5AAAAIHUWJI+SrI/WYeuw
M8gNH2FfJaG35tzlg4u7C+XYx/tuAAAAoKGY94G0a5NvWfhd0jhj6eTiai5wppIFYFm2RQ
d9S0tOcKVJfRX3igP3PyqWTlnsydTm5SuJZI5jDJKGuFVXfMfg0DioDiXv1IK5e2iWT5gb
i7xYCYlYPBxEh1DuKRxrY/+etdB0ZEDhJ7yYJxgF+YVIGSlRdLjrK7kTZT2oXfDg5Zb2lQ
tBr65anJo238V051MMeIXBsbgyt+eb05IGDwE=
-----END OPENSSH PRIVATE KEY-----

passphrase is 123

public key, name it "test-vorta-issue.pub" (chmod 644):

ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIHUWJI+SrI/WYeuwM8gNH2FfJaG35tzlg4u7C+XYx/tu andrea@andromeda

Clean the identities known to your ssh-agent: ssh-add -D should return All identities removed.

Steps to reproduce with vorta flatpak

Verify that the ssh-agent doesn't have any identities: ssh-add -l should return The agent has no identities.

Verify the KDE Wallet Manager doesn't have the private key file passphrase stored under ksshaskpass

Launch vorta flatpak and add a new profile test-vorta-issue.

Click + next to the Repository dropdown to add an Existing repository.

URL: ssh://v28lu6o4@v28lu6o4.repo.borgbase.com/./repo Repo name: test-vorta-issue Password: 123123123

In the advanced tab, leave SSH key to Automatically choose

Click the Add button: error happens Unable to add your repository and also in the main window status bar ERROR: Connection closed by remote host. Is bork working on the server?

Correct behavior with vorta (RPM)

Follow the exact same steps as the previous section, but this time with vorta RPM or any other packaging format that isn't flatpak/appimage/snap.

When clicking the Add button, no error happens. Instead you are prompted, as expected, for the ssh private key passphrase by the ksshaskpass UI, as per attachment. This is where the flatpak package would have failed. Flatpak vorta is only able to unlock the private key if it's already loaded in the ssh-agent, but if it isn't, it fails. Vorta "RPM" is able to trigger ksshaskpass which in turn adds it to the ssh-agent.

Enter 123 and click the checkbox to Remember password

image

Repo addition is successful.

ssh-add -l now shows that the correct identity has been added to the agent:

256 SHA256:4Q7s4689SP1C/AsrmNxaoT3ZPghFDGSk2XpQezN7yYU andrea@andromeda (ED25519)

Conclusion

Since the only difference is the packaging, the issue IMO lies in the flatpak version, that isn't triggering the ksshaskpass prompt.

andreaippo commented 4 months ago

Well I may have an idea of what's going on.

According to:

https://pointieststick.com/2024/04/26/this-week-in-kde-megabytes-and-gigabytes-for-all/

Implemented support for the org.freedesktop.impl.portal.Secret portal for KWallet, which lets Flatpak apps use it (Nicolas Fella, Frameworks 6.2. Link)

So it seems that this issue is caused by that.

I'll close this issue for now, as I'm confident things will work out fine once that KWallet update is released.

andreaippo commented 4 months ago

Reopening this, as I just tested again with KDE Frameworks 6.2 but the issue persists following the steps to reproduce above.

I dunno if the flatpak app's manifest file should include more permissions to talk to this "Secret Portal":

https://docs.flatpak.org/fr/latest/portal-api-reference.html#gdbus-org.freedesktop.portal.Secret

m3nu commented 4 months ago

Not using Flatpak, but we can add this permission or make other updates, if needed. How to best proceed here?

Hofer-Julian commented 4 months ago

You shouldn't need to poke holes in the sandbox in order to talk to portals. To my knowledge that's the whole point of portals. Maybe a KDE person can help here?

xalt7x commented 3 months ago

Tested Flatpak on KDE Plasma 6.1 (Fedora 41/Rawhide) . Unfortunately, the issue is still reproducible following @andreaippo steps. With native/distro packages Vorta triggers /usr/libexec/openssh/gnome-ssh-askpass , /usr/bin/ksshaskpass or whatever else was set for SSH_ASKPASS env variable. That doesn't happen with Flatpak.

However, once I pass the key(-s) to ssh-agent, Vorta Flatpak works for me.

Here's a simple workaround for KDE Plasma users:

  1. Check if you have ksshaskpass package installed on your system
  2. Check if adding key(-s) to the ssh-agent helps with Vorta Flatpak
    export SSH_ASKPASS=ksshaskpass
    grep -slR "PRIVATE" ~/.ssh/ | xargs ssh-add
  3. Create autostart file that will run the script above at the user login. ~/.config/autostart/ksshaskpass.desktop
    [Desktop Entry]
    Type=Application
    Name=ksshaskpass
    Exec=bash -c 'if [ -f /usr/bin/ksshaskpass ] && [ $XDG_CURRENT_DESKTOP = KDE ] ; then export SSH_ASKPASS=ksshaskpass ; grep -slR "PRIVATE" ~/.ssh/ | xargs ssh-add; fi'
    StartupNotify=false
    OnlyShowIn=KDE
andreaippo commented 3 months ago

Hi @xalt7x , if you're facing the same on gnome, I suppose that this is less of a KDE issue, and more of a vorta (as flatpak) or flatpak2ssh issue, WDYT?

Maybe we could check with another flatpak app needing access to ssh keys, I'm thinking an ssh client for example... If it works, the vorta flatpak maintainer could take inspiration from there (aka Ctrl C/V 😄)

xalt7x commented 3 months ago

Hi @andreaippo

if you're facing the same on gnome, I suppose that this is less of a KDE issue

I've checked GNOME (on Fedora Silverblue). Somehow it works there and even shows some native dialog (not the one from /usr/libexec/openssh/gnome-ssh-askpass which doesn't even exists on a system at the moment).

image

github-actions[bot] commented 4 days ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.