Closed petaramesh closed 1 year ago
aryoda
commented
1 year ago THX for your detailled description! Could you please append a screen shot of the settings dialog with the greyed-out "Save password to Keyring" checkbox (= 1st-time opening of the settings dialog).
PLEASE ANONYMIZE HOST-ADRESS (UNLESS IT SHOWS LOCALHOST ;-)
I need this to try to reproduce the behaviour in virtual machine...
aryoda
commented
1 year ago There is only and exactly one line of code where the checkbox "Save password to Keyring" is enabled and disabled: When __init__ of the settingsdialog.py is called:
The DEBUG output with the "keyringSupported" status is called from several code locations during the GUI startup (in this order):
app.py: mount.__init__() > password.__init__ (first log output)app.py: btnSettingsClicked > settingsdialog.__init__ (when clicking the "settings" button)app.py: btnSettingsClicked > settingsdialog.__init__ > settingsdialog.updateProfiles() > password.__init__According to the DEBUG log only the first call returned False, the second and third True and I don't understand this because call 1. does NOT call setEnabled but 2. and 3. (which are True and should enable the checkbox).
I need to set up a VM for this (sorry, this may take some time due to higher prioritization of other issues ATM)...
petaramesh
commented
1 year ago Here comes the screenshot with some greyed out personal information, and debug info in the background
As you can see, although the “save” box is greyed out, the (concealed) password appears, so it seems that Backintime was able to fetch it somehow...
aryoda
commented
1 year ago @petaramesh Perfect, THX a lot (esp. for the complete DEBUG log with the code line numbers)!
petaramesh
commented
1 year ago If I immediately close the config dialog (cancel) and open it again, this time the “save passphrase” box appears both available and already checked, and the debug list shows only one
« DEBUG: [common/tools.py:827 keyringSupported] Found appropriate keyring 'keyring.backends.kwallet' »
line.
Note that in the meantime, a cronned snapshot has run successfully without prompting for a passphrase...
aryoda
commented
1 year ago Which OS distro and version are you using (allows me to pick other issues for the same version to be debugged in the same VM)?
Which python version is used?
petaramesh
commented
1 year ago I'm using here a Manjaro (Arch based) ARM “aarch64” on a Pinebook Pro (Rockchip RK3399 ARM SOC).
So that's no x86 architecture, you may have hard times getting the exact same setup in Virtualbox, but an x86_64 Manjaro should be close enough ;)
Or you may have a Raspberry Pi 4 somewhere ?
Python is 3.10.7.
All Manjaro packages are most recent and up-to-date.
BTW I compiled Backintime on my machine from Arch's AUR, and during build, the “make tests” systematically failed, so I had to remove the “make tests” phase from the build process, in order to get a Backintime package - that actually works fine except for the bugs that I reported.
About the tests that failed, some seemed to be timeouts that may be related to the fact that this ARM machine is relatively slow, some other failed tests seemed to be rsync, which I now possibly relate to the rsync version issue.
Thanks for all your kind help.
aryoda
commented
1 year ago OT: @petaramesh Interesting, a Pinebook Pro. I am watching the PinePhone for a longer time now but hesitate since I have the Purism Phone and are not happy with it (neither OS and App landscape nor the really bad battery lifetime). I wanted to implement or port some important but missing Linux Apps to Purism and Pine Phone but realized this would require me 2 more life times ;-)
petaramesh
commented
1 year ago OT: Well I'm really happy with my (new) Pinebook Pro - besides the fact that's it's obviously not a power horse. The only shortcomings are missing a few commercial apps that are not ported to the aarch64 architecture. The one that I miss most being pCloud... The other shortcomings are the impossibility to get S3 suspend with a NVMe - some firmware/kernel issue remaining, hopefully fixed someday, some hardware issue causing quite a loud parasitic buzz in the audio output, and the stupid 1.2 meter (4 feet !) power cable that is a real shame. For the rest that's a definitely cool toy :) (And the bluetooth works better than in most of my x86 machines)
buhtz
commented
1 year ago aarch64
Side question from my side.
Is aarch64 a different architecture then armhf, arm64 and armel? Or is one of them a synonym?
All 4 of them are of the ARM family, right?
aryoda
commented
1 year ago The ARM family has huge diversity of supported instructions sets and features (like with or without floating point unit) and I honestly don't know the exact details but each many mobile phones and embedded devices use them because you can license and customize them according to your requirements.
Debian lists the supported architectures with some comments about the differences here:
aryoda
commented
1 year ago OT @buhtzz If you want to learn and understand the basic differences between not only ARM but also x86 and other architectures (edit: from the Assembly language side) there is a very good and short introduction in this book (chapter 2)- the ebooks are frequently offered for about 5 EUR/USD at special occasions so don't pay the full price ;-)
https://www.packtpub.com/product/mastering-malware-analysis-second-edition/9781803240244
petaramesh
commented
1 year ago “aarch64” is the official name of the ARM 64-bits processors instruction set (called by Debian “Arm64” as it seems). The ARM processor family is fundamentally different from the “x86” PC processor familiy. More on Wikipedia about aarch64 More on Wikipedia about ARM
aryoda
commented
1 year ago DEBUG: [common/tools.py:829 keyringSupported] No appropriate keyring found. 'keyring.backends.chainer' can't be used with BackInTime
aryoda
commented
1 year ago Related to
See also:
keyring: https://pypi.org/project/keyring/keyring: https://github.com/jaraco/keyring/issues/496aryoda
commented
1 year ago OK, I think
keyring andkeyring picks the wrong backend (not KWallet) since keyring can not exactly know which backend the user wants to use
I cannot see a keyring.set_backend() call in the code which could select the backend the user uses on the system...Why it works when the "open settings dialog" is opened again (2nd time) I don't understand yet...
Anyhow there could be a work-around with a config file to specify the used backend as default:
Open a terminal and list the available backends:
keyring --list-backends
In my case (Ubuntu 20.04.05) it shows:
keyring.backends.kwallet.DBusKeyring (priority: 4.9)
keyring.backends.SecretService.Keyring (priority: 5)
keyring.backends.chainer.ChainerBackend (priority: 10)Find out the correct path for the config file:
python3 -c "import keyring.util.platform_; print(keyring.util.platform_.config_root())"
Shows eg. /home/username/.local/share/python_keyring
Create a config file named keyringrc.cfg for the keyring python package with the default backend chosen from 1., eg.:
[backend]
default-keyring=keyring.backends.kwallet.DBusKeyring(Re)start the BiT GUI and test it.
Perhaps a backintime pw-cache stop may be required first...
For a sustainable solution I'd like to implement what @jaraco proposed here: https://github.com/jaraco/keyring/issues/391#issuecomment-625361282 I think it would be good to add a way for users to choose the "secrets management" backend in the settings GUI instead of forcing the user to create a settings files which is quite obscur (intransparent)...
@petaramesh Could you please try to create such a config file and report here if it helped?
Edit: Your default backend keyring.backends.kwallet is not supported by BiT but for kwallet you could use these backends instead:
petaramesh
commented
1 year ago Hello,
Thanks for all the help.
Actually, after I had read other bug reports and forum entries, and before reporting this one, I had already created :
❯ cat .config/python_keyring/keyringrc.cfg [backend] default-keyring=keyring.backends.kwallet.DBusKeyring
Without this, it wouldn't work at all
With this, it exhibits this strange “works on 2nd attempt” behaviour. BTW the “2nd attempt” thing concerns only the configuration dialog, besides this, i.e. cronned backups seem to work just fine.
As per : ❯ keyring --list-backends keyring.backends.libsecret.Keyring (priority: 4.8) keyring.backends.SecretService.Keyring (priority: 5) keyring.backends.chainer.ChainerBackend (priority: 10) keyring.backends.kwallet.DBusKeyring (priority: 5.1) keyring.backends.fail.Keyring (priority: 0)
For the record, my wife has an x86 Deepin 20.7 Linux machine (so a rather different distro, more or less debian-based, with DDE which is a QT-based GUI) with Backintime 1.1.24, python 3.7.3 and python 2.7.16.
On this machine, I also can save the SSH passphrase only if I create the exact same .config/python_keyring/keyringrc.cfg, but on this machine it doesn't show the strange “works only the 2nd time” issue.
As per you suggestion of creating a “choose secret management” dialog, Sirikali (Disk encryption GUI that supports cryptsetup, gocryptfs, ecryptfs, cryfs and more) has this exact feature ; you might like to take a peek at its code ?
Kind regards.
(Sirikali dialog for selecting keyring attached)
aryoda
commented
1 year ago had already created : ❯ cat .config/python_keyring/keyringrc.cfg
I already forgot that you have put this info into your issue description :smile:
THX, Sirikali is a great reference (just looking into the CPP code there not only for the GUI but also for the implementation)...
I am currently preparing a PR with more DEBUG output in the BiT keyring detection function to get more info what is going on in the background since I still cannot reproduce the issue despite it is obviously there (perhaps a DBus issue on slow computers)... It would be great if you could run this code then on your computer then...
petaramesh
commented
1 year ago Well, I don't have a feeling it's a matter of speed, the behaviour is perfectly reproductible even when the computer is basically doing nothig and there's plenty of RAM available. It's this triple, contradictory message that puzzles me :
DEBUG: [common/tools.py:829 keyringSupported] No appropriate keyring found. 'keyring.backends.kwallet' can't be used with BackInTime DEBUG: [common/tools.py:827 keyringSupported] Found appropriate keyring 'keyring.backends.kwallet' DEBUG: [common/tools.py:827 keyringSupported] Found appropriate keyring 'keyring.backends.kwallet'
As it first always says that 'keyring.backends.kwallet' can't be used with BackIntime, and just afterwards that it's appropriate...
Of course I wish to help, so if you throw some code I could test I believe I will figure out how to compile it using existing AUR's PKGBUILD... (Although I will have very litle availability from this evening until tuesday, at best).
BTW, on my wife's Deepin machine that doesn't have the issue, I get : ❯ keyring --list-backends keyring.backends.SecretService.Keyring (priority: 5) keyring.backends.kwallet.DBusKeyring (priority: 4.9) keyring.backends.kwallet.DBusKeyringKWallet4 (priority: 3.9) keyring.backends.chainer.ChainerBackend (priority: 10) keyring.backends.fail.Keyring (priority: 0)
And when opening the config dialog for the first time, I get 3 DEBUG outputs that read : DEBUG: [common/tools.py:616 keyringSupported] Found appropriate keyring 'keyring.backends.kwallet' DEBUG: [common/tools.py:616 keyringSupported] Found appropriate keyring 'keyring.backends.kwallet' DEBUG: [common/tools.py:616 keyringSupported] Found appropriate keyring 'keyring.backends.kwallet'
So you see that I get 3 successes instead of 1 failure then 2 successes on my machine ; also the code line numbers are different so I assume the versions are different (Backintime 1.1.24, python 3.7.3 and python 2.7.16 on my wife's machine)
Both machines agree on using 'keyring.backends.kwallet' anyway.
On my machine, the question seems to be « Why is line 829 unhappy and line 827 happy ? » ;-)
aryoda
commented
1 year ago On my machine, the question seems to be « Why is line 829 unhappy and line 827 happy ? » ;-)
Perfect summary :-) And luckily I can reproduce this behavior now on one machine.
It looks like that for some keyring versions the metadata is no longer pre-loaded but only if something
happens internally for the first time (eg. calling keyring.get_keyring()).
Of course I wish to help, so if you throw some code I could test I believe I will figure out how to compile it using existing AUR's PKGBUILD
THX for your offer to test this! There is no need to repackage BiT, I have prepared a standalone python script to test this:
Please download the file and rename it to *.py (since Github does not allow to upload py files) - make sure nobody has injected malicious code ;-)
Then run the code with: python3 keyringCheck.py
In my case the debug output clearly shows the same behavior (excerpt):
DEBUG: [common/keyringCheck.py:14 <module>] keyring version: 23.9.1
First call of keyringSupported()
DEBUG: [common/keyringCheck.py:68 keyringSupported] Available supported backends: []
DEBUG: [common/keyringCheck.py:73 keyringSupported] No appropriate keyring found. 'keyring.backends.chainer' can't be used with BackInTime
Second call of keyringSupported()
DEBUG: [common/keyringCheck.py:68 keyringSupported] Available supported backends: [<class 'keyring.backends.SecretService.Keyring'>, <class 'keyring.backends.kwallet.DBusKeyring'>]
DEBUG: [common/keyringCheck.py:73 keyringSupported] No appropriate keyring found. 'keyring.backends.chainer' can't be used with BackInTimeOne difference on this machine is that I am using chainer as backend by default but this is (not yet) supported by BiT
even though I think it should because it iterates over all installed backends (no need to create a config file then I think until there is a conflict like the same key is stored in two or more backends with different credentials/secrets...)
OT: In fact keyring currently totally ignores my config file for unknown reason (TODO)...
Please edit the py file and uncomment the line 33 (# keyring.get_keyring()),
then rerun with python3 keyringCheck.py
The available keyring backends should then be recognized already in the first call (as on my machine):
DEBUG: [common/keyringCheck.py:14 <module>] keyring version: 23.9.1
First call of keyringSupported()
DEBUG: [/keyringCheck.py:68 keyringSupported] Available supported backends: [<class 'keyring.backends.SecretService.Keyring'>, <class 'keyring.backends.kwallet.DBusKeyring'>]
DEBUG: [/keyringCheck.py:73 keyringSupported] No appropriate keyring found. 'keyring.backends.chainer' can't be used with BackInTime
Second call of keyringSupported()
DEBUG: [/keyringCheck.py:68 keyringSupported] Available supported backends: [<class 'keyring.backends.SecretService.Keyring'>, <class 'keyring.backends.kwallet.DBusKeyring'>]BTW: If I use keyring version 18.0.1 it also works immediately (without any fix) so I think it is a keyring issue
petaramesh
commented
1 year ago Hi,
Your test script crashes on me with :
❯ python3 keyringCheck.py First call of keyringSupported() Traceback (most recent call last): File "~/tmp/keyringCheck.py", line 41, in keyringSupported try: backends.append(keyring.backends.SecretService.Keyring) AttributeError: module 'keyring.backends' has no attribute 'SecretService'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "~/tmp/keyringCheck.py", line 85, in
I tried to comment out the SecretService lines to check for other keyring backends, but it crashes on every one it tries, in turn.
aryoda
commented
1 year ago Your test script crashes on me with :
Sorry, my bad, the logger module source code of BiT was still used. I have replaced the logger output by print statements, it should work now (see attached file)
python3 keyringCheck.txt is working too on my computer (surprisingly :-)
petaramesh
commented
1 year ago Looks bettter now, that's whhat I get :
First run :
❯ python3 keyringCheck.py keyring version: 23.9.3 keyring config file location: ~/.config/python_keyring keyring config file exists: True First call of keyringSupported() Metaclass keyring.backends.SecretService.Keyring not found: AttributeError("module 'keyring.backends' has no attribute 'SecretService'") Metaclass keyring.backends.Gnome.Keyring not found: AttributeError("module 'keyring.backends' has no attribute 'Gnome'") Metaclass keyring.backends.kwallet.Keyring not found: AttributeError("module 'keyring.backends' has no attribute 'kwallet'") Metaclass keyring.backends.kwallet.DBusKeyring not found: AttributeError("module 'keyring.backends' has no attribute 'kwallet'") Metaclass keyring.backend.SecretServiceKeyring not found: AttributeError("module 'keyring.backend' has no attribute 'SecretServiceKeyring'") Metaclass keyring.backend.GnomeKeyring not found: AttributeError("module 'keyring.backend' has no attribute 'GnomeKeyring'") Metaclass keyring.backend.KDEKWallet not found: AttributeError("module 'keyring.backend' has no attribute 'KDEKWallet'") Available supported backends: [] No appropriate keyring found. 'keyring.backends.kwallet' can't be used with BackInTime Second call of keyringSupported() Metaclass keyring.backends.SecretService.Keyring not found: AttributeError("module 'keyring.backends' has no attribute 'SecretService'") Metaclass keyring.backends.Gnome.Keyring not found: AttributeError("module 'keyring.backends' has no attribute 'Gnome'") Metaclass keyring.backends.kwallet.Keyring not found: AttributeError("module 'keyring.backends.kwallet' has no attribute 'Keyring'") Metaclass keyring.backend.SecretServiceKeyring not found: AttributeError("module 'keyring.backend' has no attribute 'SecretServiceKeyring'") Metaclass keyring.backend.GnomeKeyring not found: AttributeError("module 'keyring.backend' has no attribute 'GnomeKeyring'") Metaclass keyring.backend.KDEKWallet not found: AttributeError("module 'keyring.backend' has no attribute 'KDEKWallet'") Available supported backends: [<class 'keyring.backends.kwallet.DBusKeyring'>] Found appropriate keyring 'keyring.backends.kwallet'
Second run after uncommenting line # 33
❯ python3 keyringCheck.py keyring version: 23.9.3 keyring config file location: ~/.config/python_keyring keyring config file exists: True First call of keyringSupported() Metaclass keyring.backends.SecretService.Keyring not found: AttributeError("module 'keyring.backends' has no attribute 'SecretService'") Metaclass keyring.backends.Gnome.Keyring not found: AttributeError("module 'keyring.backends' has no attribute 'Gnome'") Metaclass keyring.backends.kwallet.Keyring not found: AttributeError("module 'keyring.backends.kwallet' has no attribute 'Keyring'") Metaclass keyring.backend.SecretServiceKeyring not found: AttributeError("module 'keyring.backend' has no attribute 'SecretServiceKeyring'") Metaclass keyring.backend.GnomeKeyring not found: AttributeError("module 'keyring.backend' has no attribute 'GnomeKeyring'") Metaclass keyring.backend.KDEKWallet not found: AttributeError("module 'keyring.backend' has no attribute 'KDEKWallet'") Available supported backends: [<class 'keyring.backends.kwallet.DBusKeyring'>] Found appropriate keyring 'keyring.backends.kwallet' Second call of keyringSupported() Metaclass keyring.backends.SecretService.Keyring not found: AttributeError("module 'keyring.backends' has no attribute 'SecretService'") Metaclass keyring.backends.Gnome.Keyring not found: AttributeError("module 'keyring.backends' has no attribute 'Gnome'") Metaclass keyring.backends.kwallet.Keyring not found: AttributeError("module 'keyring.backends.kwallet' has no attribute 'Keyring'") Metaclass keyring.backend.SecretServiceKeyring not found: AttributeError("module 'keyring.backend' has no attribute 'SecretServiceKeyring'") Metaclass keyring.backend.GnomeKeyring not found: AttributeError("module 'keyring.backend' has no attribute 'GnomeKeyring'") Metaclass keyring.backend.KDEKWallet not found: AttributeError("module 'keyring.backend' has no attribute 'KDEKWallet'") Available supported backends: [<class 'keyring.backends.kwallet.DBusKeyring'>] Found appropriate keyring 'keyring.backends.kwallet'
aryoda
commented
1 year ago No appropriate keyring found. 'keyring.backends.kwallet' can't be used with BackInTime Second call of keyringSupported() vs. Found appropriate keyring 'keyring.backends.kwallet' Second call of keyringSupported()
THX a lot, this reproduces the behavior on your computer
The fix should be a one-liner and you could edit the original code file from the package manager:
You need to find and edit the file $BiT_Installation_Root/common/tools.py on your computer
(I have used the paths from my Manjaro VM here):
cd /usr/share/backintime/common
sudo cp tools.py tools.py.bak
kate tools.py # without sudo - kate will ask for permissions when saving the changes...
# Goto line 807 with Strg+G
# Insert a new line between the lines 806 and 807 and enter the fix `keyring.get_keyring()`:
# The code here should then look like this:
def keyringSupported():
if keyring is None:
logger.debug('No keyring due to import error.')
return False
keyring.get_keyring()
backends = []
...
# No save the file in kate (which askes for the sudo password)
# start `backintime-qt --debug` and see if it works now...Hi, It did fix it, many thanks ! Now when I enter the config dialog, I get the “save passphrase” available and checked on the first attempt :) Debug now shows : DEBUG: [common/tools.py:828 keyringSupported] Found appropriate keyring 'keyring.backends.kwallet' DEBUG: [common/tools.py:828 keyringSupported] Found appropriate keyring 'keyring.backends.kwallet' DEBUG: [common/tools.py:828 keyringSupported] Found appropriate keyring 'keyring.backends.kwallet'
Many thanks for all your great help !
aryoda
commented
1 year ago Happy to hear that it works now :smiley:
Couldn't have done it without your precise descriptions and support!
I will prepare a bug fix PR for the next BiT version and perhaps a patch file for package maintainers who want to back-port this fix (since it is so easy).
Have nice weekend!
petaramesh
commented
1 year ago Nice week-end to you too, many thx again !
aryoda
commented
1 year ago Assuming the keyring backend configuration is correct (existing default backend)
the reason for this problems seems to be that a optimization in keyring
did no longer pre-load the metaclasses of internal backends
so that the whitelist-check in BiT (tools.keyringSupported()) could not build
the list of supported backends.
See: TODO Add link to issue/code change at keyring (couldn't find it)
Terminal commands:
# Show default backend
python3 -c "import keyring.util.platform_; print(keyring.get_keyring().__module__)"
# List available backends:
keyring --list-backends
# Get config location:
python3 -c "import keyring.util.platform_; print(keyring.util.platform_.config_root())"
# Create config file named "keyringrc.cfg" with one of the available backends (listed above)
[backend]
default-keyring=keyring.backends.kwallet.DBusKeyring
# new feature since Sept, 2022 (post version 1.3.2)
backintime --diagnosticsI will just move the keyring.get_keyring() call before the whitelist-check of the supported backends in
so that I don't have to call keyring.core.init_backend explicitly (since this is done internally for the first call of get_keyring() anyhow) as @jaraco recommended here: https://github.com/jaraco/keyring/issues/391#issuecomment-625361282
Just moving code has no impact on the used (and assumed) API.
I will also add more logger.debug output to improve diagnosing future issues with keyring (but only in the PR, not in the patch file).
init_backend(limit=None) of keyring takes an optional limit (filter) argument which can
take a callable "filter function" with the signature shall_the_backend_be_added_to_the_chain(backend): Bool.
It sets the current backend to an instance of the marker backend fail.Keyring if no backend passed the filter
which could be used by BiT to report "no supported keyring backend available".
This requires keyring >= 17.1.
https://github.com/jaraco/keyring/blob/bfa0148ab43ea9e592232c561971bf06712b625b/keyring/core.py#L101
keyring.backends.chainer.ChainerBackend shall be added to the supported backends too.
It is a meta backend that interates over all other installed backends. Since it seems to be the default in keyring
it would always require to add a keyring config file manually (bad user experience ;-)
On the other hand ChainerBackend does not guarantee that a BiT-supported backend is really found and used.
So personally I'd prefer to either add a checkbox in the settings GUI to enable the Chainer or even better add add
a backend selector into the settings GUI.
See also section "alternatives to fix"...aryoda
commented
1 year ago Here comes a patch with only the minimal changes required as HOTFIX and possible BACKPORT to v1.2.1.
The issue itself is fixed in the master branch with the above mentioned (already merged) PR.
USE THE PATCH AT YOUR OWN RISK (NOT DEEPLY TESTED)!
To apply the fix on the cloned git version use the attached patch file in the root folder of the git repo:
patch -p0 -i patch_issue_1321_hotfix.diff.txt
Note: This patch can also be applied to version 1.2.1 which is the version packaged for Ubuntu 20.04 (the current Git version as of today is 1.3.2). You just have to patch or manually modify the installed file of the packaged version of BiT (for Ubuntu: in /usr/share/backintime/common/tools.py).
patch_issue_1321_hotfix.diff.txt
BTW: The patch file has the suffix *.txt to be able to upload it here at Github into the comment...
Hello,
I use Backintime Version: 1.3.2 with KDE plasma.
At first the “Save SSH key password” box was greyed out, and I found here the suggestion to create a :
~/.config/python_keyring/keyringrc.cfg containing : [backend] default-keyring=keyring.backends.kwallet.DBusKeyring
But now it works rather wierdly. The 1st time I open the configuration dialog, Backintime debug prints : DEBUG: [common/tools.py:829 keyringSupported] No appropriate keyring found. 'keyring.backends.kwallet' can't be used with BackInTime DEBUG: [common/tools.py:827 keyringSupported] Found appropriate keyring 'keyring.backends.kwallet' DEBUG: [common/tools.py:827 keyringSupported] Found appropriate keyring 'keyring.backends.kwallet'
...But the “save password” checkbox stays greyed out.
Now if I exit the configuration dialog and open it again, Backintime debug prints : DEBUG: [common/tools.py:827 keyringSupported] Found appropriate keyring 'keyring.backends.kwallet' (Only once)
...And now I get the proper “save password” checkbox, and it works.
Looks like a bug to me.