Closed lilyLuLiu closed 1 month ago
lilyLuLiu
commented
2 months ago @anjannath made some investigation on the machine:
crc makes use of the secrets service of dbus on linux to store the pull-secret, but to unlock the secrets collection it needs a password, from a GUI you'll see a dialog asking you to enter the password, but while on a SSH session this is not possible so its stuck.
praveenkumar
commented
2 months ago @anjannath does this happen when pull-secret-file is used as part of config?
anjannath
commented
2 months ago @anjannath does this happen when
pull-secret-fileis used as part of config?
this happens early on when we want to initialize the config object, so this happens even when pull-secret-file is set, when executing this line: https://github.com/crc-org/crc/blob/4e80c4c48c6c0c13d0399cc0436a0a38d9dda0a2/cmd/crc/cmd/root.go#L152
this tries to determine if the secrets store is accessible by trying to store a value, but if the login secrets collection is locked, and crc is being run in an ssh session then it gets stuck, because the prompt to unlock the keyring is a GUI prompt
albfan
commented
2 months ago Agreed with @praveenkumar. Even if we want to triage and resolve this (crc stuck with accessing pull request) which probably is just timeout and warn about the problem (no access to secret service: Something for user to resolve) --pull-secret-file is a valid workaround, as we don't want any prompt for a non interactive session.
anjannath
commented
2 months ago This'll be stuck even when we use the --pull-secret-file flag or set it before hand using crc config set pull-secret-file
from crc we should at least let the user know that the secret service is not accessible and should not block forever when failing to access the keyring
we hit this blocking issue when checking if the keyring is accessible, https://github.com/crc-org/crc/blob/4e80c4c48c6c0c13d0399cc0436a0a38d9dda0a2/pkg/crc/config/secret_config.go#L68
@albfan you mentioned using busctl instead to try to access the keyring and determine if its accessible, i think that'll solve this issue
albfan
commented
2 months ago I check documentation and a collection may or may not ask for a prompt
I did a quick test and it always ask me for prompt:
$ cat keyring-create.py
#!/usr/bin/env python
from pydbus import SessionBus
from gi.repository import GLib
collection_name = "crc-test"
properties = {"org.freedesktop.Secret.Collection.Label": GLib.Variant.new_string(collection_name)}
ses_bus = SessionBus()
service_name = 'org.freedesktop.secrets'
secret_service = ses_bus.get(service_name, '/org/freedesktop/secrets')
mainloop = GLib.MainLoop()
def _received_pw(dismissed, object_path):
print("dismissed?", dismissed, object_path)
mainloop.quit()
def show_prompt(prompt_id):
prompt = ses_bus.get(service_name, prompt_id)
prompt.onCompleted = _received_pw
prompt.Prompt("random_id_for_window")
mainloop.run()
print('Prompt closed')
def add_my_collection():
result = secret_service.CreateCollection(properties, "")
print("result from CreateCollection", result)
if result[1] != '/':
show_prompt(result[1])
def main():
add_my_collection()
if __name__ == '__main__':
main()$ cat keyring-list.py
#!/usr/bin/env python
from pydbus import SessionBus
from gi.repository import GLib
collection_name = "MyTestCollection"
properties = {"org.freedesktop.Secret.Collection.Label": GLib.Variant.new_string(collection_name)}
ses_bus = SessionBus()
service_name = 'org.freedesktop.secrets'
secret_service = ses_bus.get(service_name, '/org/freedesktop/secrets')
mainloop = GLib.MainLoop()
def list_collections():
print('print collection names')
for test_collect in secret_service.Collections:
print(test_collect)
def main():
list_collections()
if __name__ == '__main__':
main()$ cat keyring-delete.py
#!/usr/bin/env python
from pydbus import SessionBus
from gi.repository import GLib
collection_name = "crc-test"
properties = {"org.freedesktop.Secret.Collection.Label": GLib.Variant.new_string(collection_name)}
ses_bus = SessionBus()
service_name = 'org.freedesktop.secrets'
secret_service = ses_bus.get(service_name, '/org/freedesktop/secrets')
mainloop = GLib.MainLoop()
def _received_pw(dismissed, object_path):
print("dismissed?", dismissed, object_path)
mainloop.quit()
def show_prompt(prompt_id):
prompt = ses_bus.get(service_name, prompt_id)
prompt.onCompleted = _received_pw
prompt.Prompt("random_id_for_window")
mainloop.run()
print('Prompt closed')
def add_my_collection():
result = secret_service.CreateCollection(properties, "")
print("result from CreateCollection", result)
if result[1] != '/':
show_prompt(result[1])
def remove_my_collection():
print('print collection names')
for test_collect in secret_service.Collections:
print(test_collect)
#if collection_name in test_collect:
if True:
print('deleting collection')
this_collection = ses_bus.get(service_name, test_collect)
result = this_collection.Delete()
print(result)
if result != '/':
show_prompt(result)
def main():
#add_my_collection()
remove_my_collection()
if __name__ == '__main__':
main()but after create two same collections, they get a different suffix:
$ ./keyring-create.py
result from CreateCollection ('/', '/org/freedesktop/secrets/prompt/p25')
dismissed? False /org/freedesktop/secrets/collection/crc_2dtest
Prompt closed
[alberto@fedora crc]$ ./keyring-create.py
result from CreateCollection ('/', '/org/freedesktop/secrets/prompt/p26')
dismissed? False /org/freedesktop/secrets/collection/crc_2dtest_5f1
Prompt closed
[alberto@fedora crc]$ ./keyring-list.py
print collection names
/org/freedesktop/secrets/collection/crc_2dtest_5f1
/org/freedesktop/secrets/collection/crc_2dtestAnd it always ask for a prompt, so I'm not sure how crc can access a collection without prompting, or a locked collection can ask for prompt and block the crc execution
Can we patch the crc cli to add extra traces on CreateCollection or CreateItem to check when it needs a prompt and fail like this?
anjannath
commented
2 months ago And it always ask for a prompt, so I'm not sure how crc can access a collection without prompting, or a locked collection can ask for prompt and block the crc execution
for crc we are not creating a new collection, but using the pre-existing login collection (actually this is decided for us by the go module we are using)
/org/freedesktop/secrets/collection/login or /org/freedesktop/secrets/aliases/default depending on which exists.
maybe i had this wrong observation, but i remember from testing this earlier that on a GUI flow where an users types in their password to login will have the login collection automatically unlocked after successful login
cfergeau
commented
2 months ago but while on a SSH session this is not possible so its stuck.
On my headless RHEL9 machine, I get an error when it tries to access the secret:
INFO Loading bundle: crc_microshift_libvirt_4.16.0_amd64...
DEBU Cannot load secret from configuration: empty path
DEBU Cannot load secret from keyring: The name is not activatable
CRC requires a pull secret to download content from Red Hat.
You can copy it from the Pull Secret section of https://console.redhat.com/openshift/create/local.
? Please enter the pull secret Similar error with keyring-list.py:
$ python3 ./keyring-list.py
Traceback (most recent call last):
File "/home/teuf/dev/crc/./keyring-list.py", line 11, in <module>
secret_service = ses_bus.get(service_name, '/org/freedesktop/secrets')
File "/home/teuf/.local/lib/python3.9/site-packages/pydbus/proxy.py", line 44, in get
ret = self.con.call_sync(
gi.repository.GLib.GError: g-dbus-error-quark: GDBus.Error:org.freedesktop.DBus.Error.ServiceUnknown: The name is not activatable (2)If I install a package providing /org/freedesktop/secrets maybe I'll start seeing these freezes as well.
albfan
commented
2 months ago pkg/crc/cluster/pullsecret.go
const helpMessage = `CRC requires a pull secret to download content from Red Hat.https://github.com/crc-org/crc/blob/main/pkg/crc/cluster/pullsecret.go#L168
pkg/crc/cluster/pullsecret.go
53:21: pullSecret, err := promptUserForSecret()https://github.com/crc-org/crc/blob/main/pkg/crc/cluster/pullsecret.go#L53
Looks like crc detects when session is non interactive, so it cannot prompt and ask for pull-request-file directly
I suppose the hang is in other place.
@cfergeau can you check if you have this service available?
$ busctl list --user | grep secrets
org.freedesktop.secrets 2965 gnome-keyring-d cloud-user :1.6 session-4.scope 4 Using waypipe we get access to graphical interface from a ssh wayland session.
Removing current login storage and restarting it now password is know and gnome-keyring-daemon --replace --unlock works.
That's still a workaround until we identify what locks the login collection.
$ is-collection-locked login
method return time=1726572308.964656 sender=:1.1013 -> destination=:1.1028 serial=51 reply_serial=2
variant boolean true
$ echo -n "mypassword" | gnome-keyring-daemon -r --unlock
discover_other_daemon: 0** Message: 13:25:13.345: Replacing daemon, using directory: /run/user/1000/keyring
GNOME_KEYRING_CONTROL=/run/user/1000/keyring
SSH_AUTH_SOCK=/run/user/1000/keyring/ssh
$ is-collection-locked login
method return time=1726572316.108909 sender=:1.1029 -> destination=:1.1030 serial=21 reply_serial=2
variant boolean false@cfergeau can you check if you have this service available?
Hadn't seen this request before, but:
$ busctl list --user | grep secrets
$After rebuild key storage, it was not locked again. there's a log checking each second
We might close this issue, or keep open to continue tracking it
albfan
commented
1 month ago Closing as not reproducible anymore
Machine: Red Hat Enterprise Linux release 8.10 (Ootpa) The crc command stuck even for
crc versionorcrc help. If reboot the machine, crc back to normal. This happens frequently and only in this machine.