`Error: access_denied: No scopes found` on oidc-add

[DrDaveD]

When we restart oidc-agent and do oidc-add to add back an account accessing CILogon, we always get the error

Error: access_denied: No scopes found`

I looked into it and found it has to do with the fact that we have a special arrangement with CILogon to send them one scope when we do the original OIDC authentication (wlcg.capabiityset) and then they associate a list of individual scopes with that in the refresh token and access token. Then when oidc-add runs, it apparently sends in the refresh token to the token issuer along with the original set of scopes, none of which include the returned set of scopes associated with the refresh and access tokens. Even though oidc-add returns this error, the account is loaded, so it did what we wanted it to do to make ready for oidc-token, it just had this additional exchange with the token issuer which we don't want and which failed. So it isn't a fatal error, but it does a useless operation and messes things up in the logs.

The first question is, is there already a way around that? None of the oidc-add options looks promising. So if there isn't a way around that, an option be added to oidc-add to avoid this problem, either an option to avoid contacting the token issuer at all (my preference) or an option we can use to prevent it from sending the original scopes? We had a related issue with the oidc-token command in that by default it sends the original scopes as well, but we get around that by always using --scopes " ".

[zachmann]

When adding an account with oidc-add it runs one refresh flow to ensure that everything works fine. This uses the scopes configured for that account - usually that works fine. As you noted with oidc-token there is an option to request specific scopes. You could use oidc-token with the agent's autoload feature and don't load the account with oidc-add. However, I guess you have a setup where this might not be suitable.

I adapted the agent, so it does not ignore the scopes returned from the OP when an account config is created. If the OP then returns the correct set of scopes, the agent stores it in the account config and further calls to both oidc-add and oidc-token should work as expected.

Could you test the change and confirm that this fixes the issue. You can find the packages for centos 7 here: https://codebase.helmholtz.cloud/m-team/oidc/oidc-agent/-/jobs/1410591/artifacts/browse/results/centos/7/

[DrDaveD]

I will try the new version later today, but actually an autoload feature on oidc-token would probably be perfect for us. I don't see an option for that on oidc-token, however, and just running oidc-token by itself complains that there is no such account. So how do you use the autoload feature?

[zachmann]

Running oidc-token is enough. However, this requires either to use gpg encryption or oidc-prompt to be installed (comes with the oidc-agent-desktop package which is automatically installed if the oidc-agent meta-package is installed (and not only oidc-agent-cli). Then the agent will (gui-)prompt for the encryption password and load it before returning the AT to oidc-token. If autoload is not possbile (or is declined by the user) oidc-token will receive the account not loaded error.

[DrDaveD]

I see. So you're right, we can't use the autoload feature, because we're not using gpg encryption and this is for a script run from cron.

I tried version 5.0.2 but it didn't help. I still get Error: access_denied: No scopes found. when I do the oidc-add. I tried enabling debug with -g but did not see any messages.

[zachmann]

I just realized I wasn't very clear about this:

• You have to restart the agent
• You have to create a new account config with oidc-gen or re-authenticate the existing one with oidc-gen --reauthenticate

If you did this and it still does not work, I'll have to take another look. You can also look at the scope value in the account config after de-crypting it with oidc-gen -p. It should list the correct ("expanded") list.

However, this will only work if the OP returns the correct list of scopes when the RT is obtained. I don't know if this is the case or not.

[DrDaveD]

Ok. So this time I tried

eval `oidc-agent`
oidc-gen --reauthenticate -w device --manual dune

and reauthenticated, but then oidc-gen -p dune | jq .scope did not show the returned list of scopes, it showed the original list. If I then stop and start oidc-agent and do oidc-add it again gives me the No scopes found error, of course.

I haven't tried removing the account and starting from scratch, but it occurs to me that if those scopes did get replaced it would interfere with another oidc-gen --reauthenticate, which we do sometimes need to do. So maybe we need another approach.

[zachmann]

I added an option to oidc-add to directly load the config into the agent without doing the token flow. You can use it as oidc-add <shortname> --skip-check after starting the new version of the agent.

Package is available here: https://codebase.helmholtz.cloud/m-team/oidc/oidc-agent/-/jobs/1413979/artifacts/browse/results/centos/7/

[DrDaveD]

That does eliminate the error message. However when I follow that with oidc-token --scope=" " dune I get an error

Error: could not connect to url

and when I run it via the oidc-token-renewer systemd service I get a slightly longer error message:

Dec 21 09:25:28 dwdosgdev.fnal.gov oidc-agent.http[22527]: (src/oidc-agent/http/http_errorHandler.c:32) handleHost (src/oidc-agent/http/http_errorHandler.c:32) HTTPS Request failed: URL using bad/illegal format or missing URL Please check the provided URLs.
Dec 21 09:25:28 dwdosgdev.fnal.gov oidc-agent.d[22520]: (src/oidc-agent/http/http_ipc.c:24) Error from http request: could not connect to url

I did reauthenticate to make sure everything was cleared out before trying again.

Here's the end of an strace on the command:

openat(AT_FDCWD, "/var/spool/osg-token-svc/.oidc-agent/", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 3
close(3)                                = 0
open("/var/spool/osg-token-svc/.oidc-agent/config", O_RDONLY) = -1 ENOENT (No such file or directory)
access("/etc/sysconfig/strcasecmp-nonascii", F_OK) = -1 ENOENT (No such file or directory)
socket(AF_UNIX, SOCK_STREAM, 0)         = 3
connect(3, {sa_family=AF_UNIX, sun_path="/tmp/oidc-4nQPc3/oidc-agent.21314"}, 110) = 0
getrandom("\x9c\x5a\xa0\x1f\x04\xa8\xfe\x48\xcd\x10\xb8\x6f\xad\x1f\x6c\x86", 16, 0) = 16
getrandom("\xb9\x57\x80\x09\xbf\x0f\xe8\x72\x29\x86\x59\x4b\x89\xa3\xda\x4d\x18\x20\x3d\x53\x5a\x06\x45\x13\xbf\xaf\x9c\xc6\xca\x78\xda\xfa", 32, 0) = 32
fcntl(3, F_GETPIPE_SZ)                  = -1 EBADF (Bad file descriptor)
fcntl(3, F_SETPIPE_SZ, 44)              = -1 EBADF (Bad file descriptor)
write(3, "jhM5bdjD12uF3FgVnByR7Eg6rdGxvjQLRRfsZPqO1Eo=", 44) = 44
select(4, [3], NULL, NULL, NULL)        = 1 (in [3])
ioctl(3, FIONREAD, [44])                = 0
read(3, "Q9PwZRH8UL5znRLwHVrcDFsPVOtBWZGnaQ1K7fdy80M=", 44) = 44
getrandom("\x2d\x2a\x21\xd2\xda\x25\x4c\x9a\x4e\x94\x36\xae\xb8\x9c\x2b\x6b\x6c\x0e\x7a\x60\x49\xea\xce\x1e", 24, 0) = 24
fcntl(3, F_GETPIPE_SZ)                  = -1 EBADF (Bad file descriptor)
fcntl(3, F_SETPIPE_SZ, 205)             = -1 EBADF (Bad file descriptor)
write(3, "108:LSoh0tolTJpOlDauuJwra2wOemBJ6s4e:lLJlZXJBWDVAeArrOhcBx8y0qAY6OUDLPAyC9IN+NxDqSVgAe9HnfGt/JBRhiGvMtVBhFvYqumUTsZLDn80eqirnX+Qo2wk2/UuTOZ7FyUCF3ezIVXLcn8MiOAroHz/KKmW6I2SY3Q1jVDNDjTF0FgmX2uvK5DdEbtndLw==", 205) = 205
select(4, [3], NULL, NULL, NULL)        = 1 (in [3])
ioctl(3, FIONREAD, [132])               = 0
read(3, "55:SiAN4dQ/kdZAMXwk5v48V3xPd45vMk9B:EFe/fcXeEc2iCEuPzylgFeiL90IkUlh45HNrzGE7paZAuwCdPbB6cNzl9+y5Jn8ZEJYwCZc8eiRP3UJfnZlMux8SsGq0Cfc=", 132) = 132
close(3)                                = 0
ioctl(2, TCGETS, {B38400 opost isig icanon echo ...}) = 0
write(2, "\33[31mError: could not connect to url\n\33[0m", 41Error: could not connect to url
) = 41
exit_group(1)                           = ?
+++ exited with 1 +++

I'm not sure what URL it is complaining about. The oidc-gen -p configuration looks fine:

{
  "name": "dune",
  "client_name": "oidc-agent:dune",
  "issuer_url": "https://test.cilogon.org/dune",
  "mytoken_url": "",
  "config_endpoint": "https://test.cilogon.org/dune/.well-known/openid-configuration",
  "device_authorization_endpoint": "https://test.cilogon.org/oauth2/device_authorization",
  "daeSetByUser": 85899345920,
  "client_id": "REDACTED",
  "client_secret": "REDACTED",
  "refresh_token": "REDACTED",
  "cert_path": "/etc/pki/tls/certs/ca-bundle.crt",
  "scope": "org.cilogon.userinfo openid profile email wlcg.capabilityset:/dunepilot wlcg.groups:/dune wlcg.groups:/dune/pilot offline_access",
  "audience": "",
  "oauth": 0,
  "uses_pub_client": 206158430208,
  "mytoken_profile": {},
  "username": "",
  "password": ""
}

[zachmann]

Should be fixed now: https://codebase.helmholtz.cloud/m-team/oidc/oidc-agent/-/jobs/1414656/artifacts/browse/results/centos/7/

Requires to restart the agent.

[DrDaveD]

Wow that was fast! That indeed did fix it. Please let me know when a new version is tagged.

[DrDaveD]

@zachmann when do you expect the next version to be tagged?

[zachmann]

We recently moved our CI infrastructure and are currently not able to build a windows installer. We hope to have this ready until the end of the month.

We also could do a linux-only release.

[DrDaveD]

We would appreciate an earlier linux-only release.

[zachmann]

This is now available in 5.1.0 and packages are currently uploaded to our repo.

[DrDaveD]

Thanks!

[marcvs]

Sorry for the delay. I've just updated the repos. 5.1.0 is now available via the typical channels on https://repo.data.kit.edu