Permission issue for users who are part of a group

[PrasannaShasthriDevOps]

We have a group groupA which provides access to jenkins and its added in the "Azure Active Directory Matrix-based security" with enough permissions. But some users are losing permissions even though they are part of groupA and they were successfully accessed jenkins earlier. Users see below error.

userA is missing the Overall/Read permission

We have also observed bellow warnings/errors in the jenkins system logs.

2021-06-10 10:30:21.827+0000 [id=374]   SEVERE  c.m.graph.logger.DefaultLogger#logError: CoreHttpProvider[sendRequestInternal] - 386Graph service exception Error code: Request_ResourceNotFound
2021-06-10 10:30:21.827+0000 [id=374]   SEVERE  c.m.graph.logger.DefaultLogger#logError: CoreHttpProvider[sendRequestInternal] - 386Error message: Resource 'UserA' does not exist or one of its queried reference-property objects are not present.
2021-06-10 10:30:21.827+0000 [id=374]   SEVERE  c.m.graph.logger.DefaultLogger#logError: CoreHttpProvider[sendRequestInternal] - 386
2021-06-10 10:30:21.828+0000 [id=374]   SEVERE  c.m.graph.logger.DefaultLogger#logError: CoreHttpProvider[sendRequestInternal] - 386GET https://graph.microsoft.com/v1.0/users/UserA
2021-06-10 10:30:21.828+0000 [id=374]   SEVERE  c.m.graph.logger.DefaultLogger#logError: CoreHttpProvider[sendRequestInternal] - 386SdkVersion : graph-java/v3.2.0
2021-06-10 10:30:21.828+0000 [id=374]   SEVERE  c.m.graph.logger.DefaultLogger#logError: CoreHttpProvider[sendRequestInternal] - 386
2021-06-10 10:30:21.828+0000 [id=374]   SEVERE  c.m.graph.logger.DefaultLogger#logError: CoreHttpProvider[sendRequestInternal] - 386
2021-06-10 10:30:21.828+0000 [id=374]   SEVERE  c.m.graph.logger.DefaultLogger#logError: CoreHttpProvider[sendRequestInternal] - 386404 : Not Found
2021-06-10 10:30:21.828+0000 [id=374]   SEVERE  c.m.graph.logger.DefaultLogger#logError: CoreHttpProvider[sendRequestInternal] - 386[...]
2021-06-10 10:30:21.829+0000 [id=374]   SEVERE  c.m.graph.logger.DefaultLogger#logError: CoreHttpProvider[sendRequestInternal] - 386
2021-06-10 10:30:21.829+0000 [id=374]   SEVERE  c.m.graph.logger.DefaultLogger#logError: CoreHttpProvider[sendRequestInternal] - 386[Some information was truncated for brevity, enable debug logging for more details]
2021-06-10 10:30:21.829+0000 [id=374]   SEVERE  c.m.graph.logger.DefaultLogger#logError: Throwable detail: com.microsoft.graph.http.GraphServiceException: Error code: Request_ResourceNotFound
Error message: Resource 'UserA' does not exist or one of its queried reference-property objects are not present.

Version report

Jenkins and plugins versions report:

Jenkins: 2.277.4.3
---
cloudbees-analytics:1.28
next-build-number:1.6
jaxb:2.3.0.1
credentials:2.3.15.1
sonar:2.13.1
copyartifact:1.46
pipeline-graph-analysis:1.10
plain-credentials:1.7
git-server:1.9
snakeyaml-api:1.27.0
mercurial:2.14
cloudbees-unified-ui:1.6
okhttp-api:3.14.9
cloudbees-assurance:2.276.0.3
pam-auth:1.6
prisma-cloud-jenkins-plugin:21.04.412
nunit:0.27
junit:1.49
blueocean-pipeline-api-impl:1.24.6
jackson2-api:2.12.1
jdk-tool:1.5
plugin-usage-plugin:1.2
dashboard-view:2.16
docker-build-step:2.7
email-ext:2.82
configurationslicing:1.52
blueocean-dashboard:1.24.6
operations-center-analytics-config:2.222.0.1
pipeline-aggregator-view:1.11
deployer-framework:1.3
cloudbees-nodes-plus:1.22
data-tables-api:1.10.23-3
branch-api:2.6.3
github-branch-source:2.10.2
delivery-pipeline-plugin:1.4.2
blueocean:1.24.6
ssh-slaves:1.31.5
http_request:1.9.0
token-macro:2.15
git:4.7.1
code-coverage-api:1.3.2
pipeline-model-api:1.8.4
hidden-parameter:0.0.4
structs:1.22
matrix-project:1.18
badge:1.8
cloudbees-ssh-slaves:2.9
workflow-cps-checkpoint:2.10
script-security:1.76
quality-gates:2.7.2
workflow-api:2.42
workflow-cps:2.90
bouncycastle-api:2.20
operations-center-analytics-reporter:2.222.0.1
ssh-agent:1.22
envinject-api:1.7
built-on-column:1.1
promoted-builds:3.9.1
aws-credentials:1.28.1
rebuild:1.32
ssh-credentials:1.18.1
azure-ad:155.v745ce80af7ea
infradna-backup:3.38.34
ec2-fleet:2.3.2
ez-templates:1.3.4
workflow-support:3.8
blueocean-personalization:1.24.6
checks-api:1.7.0
global-build-stats:1.5
jquery:1.12.4-1
build-token-root:1.7
parameter-separator:1.3
jobConfigHistory:2.27
operations-center-context:2.277.0.5
workflow-scm-step:2.12
jdepend:1.3.0
blueocean-pipeline-scm-api:1.24.6
gradle:1.36
node-iterator-api:1.5
extended-choice-parameter:0.82
pipeline-model-extensions:1.8.4
cloudbees-bitbucket-branch-source:2.9.7
jenkins-multijob-plugin:1.36
htmlpublisher:1.25
blueocean-web:1.24.6
pipeline-multibranch-defaults:2.1
clone-workspace-scm:0.6
font-awesome-api:5.15.2-2
naginator:1.18.1
blueocean-git-pipeline:1.24.6
pipeline-maven:3.10.0
jjwt-api:0.11.2-5.143e44951c52
deployed-on-column:1.8
nexus-jenkins-plugin:3.11.20210420-142258.bdfc332
port-allocator:1.8
workflow-durable-task-step:2.38
variant:1.4
seleniumhq:0.4
groovy-postbuild:2.5
xvfb:1.1.3
cloudbees-monitoring:2.11
handlebars:1.1.1
handy-uri-templates-2-api:2.1.8-1.0
stashNotifier:1.20
blueocean-core-js:1.24.6
postbuildscript:2.11.0
blueocean-config:1.24.6
cvs:2.19
javadoc:1.6
cloudbees-long-running-build:1.16
pubsub-light:1.13
sse-gateway:1.24
m2release:0.16.2
synopsys-coverity:2.4.1
cloudbees-aborted-builds:1.14
build-with-parameters:1.5.1
pipeline-milestone-step:1.3.2
ant:1.11
config-file-provider:3.7.2
build-monitor-plugin:1.12+build.201809061734
icon-shim:3.0.0
pipeline-build-step:2.13
cloudbees-administrative-monitors:1.0.1
cloudbees-plugin-usage:2.7
cloudbees-quiet-start:1.7
cloudbees-workflow-ui:2.6
unique-id:2.2.0
favorite:2.3.3
jquery-detached:1.2.1
monitoring:1.87.0
Parameterized-Remote-Trigger:3.1.5.1
async-http-client:1.7.24.3
ivy:2.1
cloudbees-uc-data-api:4.43
pipeline-rest-api:2.19
cloudbees-template:4.49
authentication-tokens:1.4
label-linked-jobs:6.0.1
workflow-step-api:2.23
forensics-api:1.0.0
blueocean-pipeline-editor:1.24.6
plugin-util-api:2.1.0
cloudbees-jsync-archiver:5.15
emailext-template:1.2
jenkins-cloudformation-plugin:1.2
workflow-multibranch:2.23
popper-api:1.16.1-2
mask-passwords:3.0
Office-365-Connector:4.15.0
cloudbees-aws-cli:1.5.15
uno-choice:2.5.6
sidebar-link:1.12.0
zentimestamp:4.2
parameterized-trigger:2.40
ace-editor:1.1
docker-workflow:1.26
jenkins-design-language:1.24.6
mapdb-api:1.0.9.0
build-user-vars-plugin:1.7
jira:3.2.1
build-metrics:1.3
testng-plugin:1.15
support-core:2.72.1
windows-slaves:1.7
workflow-basic-steps:2.23
nested-view:1.19
pipeline-model-declarative-agent:1.1.1
cobertura:1.16
display-url-api:2.3.4
bootstrap4-api:4.6.0-3
git-validated-merge:3.30
clover:4.12.0
skip-plugin:4.10
mailer:1.34
git-parameter:0.9.13
metrics:4.0.2.7
s3:0.11.7
warnings-ng:9.0.1
blueocean-jwt:1.24.6
persistent-parameter:1.3
build-timeout:1.20
build-name-setter:2.2.0
blueocean-bitbucket-pipeline:1.24.6
workflow-remote-loader:1.5
artifactory:3.11.0
external-monitor-job:1.7
credentials-binding:1.24
browserstack-integration:1.2.3
pipeline-input-step:2.12
jquery3-api:3.6.0-1
antisamy-markup-formatter:2.1
blueocean-rest:1.24.6
pipeline-npm:0.9.2
pipeline-stage-tags-metadata:1.8.4
cloudbees-blueocean-default-theme:0.8
run-condition:1.5
echarts-api:5.0.1-1
nectar-rbac:5.57
blueocean-i18n:1.24.6
docker-traceability:1.2
cloudbees-support:3.26
custom-tools-plugin:0.8
workflow-job:2.40
durable-task:1.35
envinject:2.4.0
subversion:2.14.2
audit-trail:3.8
cloudbees-folder:6.15
basic-branch-build-strategies:1.3.2
aws-java-sdk:1.11.995
nodelabelparameter:1.8.1
nodejs:1.4.0
jira-steps:1.6.0
cloudbees-jenkins-advisor:3.2.4
ws-cleanup:0.39
saml:2.0.3
docker-custom-build-environment:1.7.3
validating-string-parameter:2.8
timestamper:1.12
hashicorp-vault-plugin:3.8.0
conditional-buildstep:1.4.1
dtkit-api:3.0.0
cloudbees-request-filter:1.7
xunit:3.0.2
blueocean-display-url:2.4.1
pipeline-utility-steps:2.8.0
ldap:2.4
matrix-auth:2.6.6
operations-center-client:2.277.0.4
github:1.33.1
view-job-filters:2.3
cloudbees-license:9.53
maven-plugin:3.10
translation:1.15
pipeline-aws:1.43
template-project:1.5.2
scm-api:2.6.4
git-client:3.7.1
momentjs:1.1.1
pipeline-stage-step:2.5
lockable-resources:2.10
h2-api:1.4.199
cloudbees-workflow-template:3.12
jobgenerator:1.22
operations-center-cloud:2.277.0.1
blueocean-autofavorite:1.2.4
global-post-script:1.1.4
dockerhub-notification:2.5.2
cloudbees-label-throttling-plugin:3.8
docker-commons:1.17
command-launcher:1.5
docker-build-publish:1.3.3
github-api:1.123
job-dsl:1.77
extensible-choice-parameter:1.7.0
blueocean-jira:1.24.6
cloudbees-even-scheduler:3.10
maven-deployment-linker:1.5.1
blueocean-commons:1.24.6
analysis-model-api:10.0.0
build-pipeline-plugin:1.5.8
jsch:0.1.55.2
blueocean-rest-impl:1.24.6
blueocean-events:1.24.6
ec2:1.56
apache-httpcomponents-client-4-api:4.5.13-1.0
workflow-aggregator:2.5
ansicolor:1.0.0
pipeline-model-definition:1.8.4
jquery-ui:1.0.2
aws-lambda:0.5.10
jacoco:3.1.1
azure-commons:1.1.3
groovy:2.4
slack:2.48
cloudbees-folders-plus:3.12
cloudbees-platform-common:1.7
blueocean-github-pipeline:1.24.6
workflow-cps-global-lib:2.18
trilead-api:1.0.13
pipeline-stage-view:2.19
nectar-license:8.31
operations-center-agent:2.277.0.2
resource-disposer:0.15

• What Operating System are you using (both controller, and any agents involved in the problem)?

OS: Linux - 4.14.173-137.229.amzn2.x86_64

Reproduction steps

Unable to reproduce as it happens randomly for some users. But once we add user directly to the azure authorisation matrix, user will be able to login without any issues.

Results

Expected result:

A Group "GroupA" which is added to the azure authorisation matrix. User who is member of this group should have access to jenkins.

Actual result:

But some users are losing permission to jenkins.

Azure AD permisions

[timja]

You can remove the Azure Active Directory Graph permissions, they aren't needed any more.

What is the actual format of 'UserA' in the error? is it their display name or their UPN (i.e. timja@my-email-domain.com?

[PrasannaShasthriDevOps]

Can you please list the graph permissions that needs to be removed ? All the permissions listed in the screenshot ?

No, its not email ID or UPN. But its "Firstname Lastname"

[timja]

Remove

Hmm I'm working on something related in https://github.com/jenkinsci/azure-ad-plugin/issues/145, it may help you

[PrasannaShasthriDevOps]

okey. thanks for the update. Looks like #145 is closed, Should I try with latest version. ?

[timja]

Yes please

[PrasannaShasthriDevOps]

While I still trying to upgrade Azure AD plugin in our test jenkins environment, I have a question..

I think the "404 : Not Found" errors that I mentioned earlier may not be related to the actual issue we are facing, where some users who are part of a group which authorise entire group members to jenkins, are losing access intermittently. We had to manually add them to azure authorisation matrix to temporarily fix the issue.

This issue happened after we upgrade Jenkins Core and azure AD plugin recently,

Jenkins Core upgraded from 2.235.5.1 to 2.277.4.3 Azure AD plugin from 1.2.1 to 155.v745ce80af7ea

[timja]

If those users go to /whoAmI you'll see what groups the plugin thinks they are in

[PrasannaShasthriDevOps]

Yeah, we have checked that earlier and they are part of the correct group. Still they can't authorise to jenkins. This is happening for only some users and across jenkins masters. Interesting thing to observe is that, if userA can't access jenkins master A, but still that user will be able to access other jenkins masters.

Also we have also observed that /whoAmI is listing both objectID and group name for a group.

[timja]

Are you able to send a screenshot of the page? annotated with what's working and what's not? It's bit hard to understand

You can email it to timjacomb1 at gmail.com

[PrasannaShasthriDevOps]

I have installed the latest version of the plugin (v172) in our test environment, but the issue seems to be persists for us. For some users even though they are part of a common group which is added in authorisation matrix, they are unable to access jenkins.

[Iridias]

We have a similar Issue since recent updates. Though our users can login, but lack the assigned permissions of the group.

Checking "/whoAmI", the specific group is not listed. I checked the group-membership in AAD, and the odd thing is: the group that is not listed, is of type "Microsoft 365", whereas all other groups are of type "Security". Is it, that the plugin only respects groups of type "Security" and no other type anymore?

[timja]

Ah right yes there was a recent change for that.

Security groups should be used here not Microsoft 365 groups.

[Iridias]

Is there an option to change that behaviour? Because otherwise, we'd have to duplicate that group and manage it manually - that would be troublesome and error-prone!

[PrasannaShasthriDevOps]

Our group which has the issue is of type "Mail enabled security", does that cause any issue ?

(optional) To enable AzureAD group support: Click Manifest and modify the "groupMembershipClaims": "None" value to "groupMembershipClaims": "SecurityGroup" manifest.

I see this in documentation and it is optional too. So we haven't changed anything in the manifest.

[timja]

I'm reverting it for now.

Mail-Enabled security groups should work.

I don't think Microsoft 365 groups should be allowed but given it worked before an option could be made for it to work.

See group types here: https://docs.microsoft.com/en-us/microsoft-365/admin/create-groups/compare-groups?view=o365-worldwide

Any thoughts?

[timja]

I see this in documentation and it is optional too. So we haven't changed anything in the manifest.

Hmm I think it will work without that since you're using the graph API here, but probably good to have it enabled anyway.

[PrasannaShasthriDevOps]

I assume, using only "Security groups" as authorities was implemented as part of https://github.com/jenkinsci/azure-ad-plugin/pull/140 (165.v36344b7d7ca7)

But we have faced the issues in Azure AD plugin version 152.v1609ed460604

Any thoughts?

[timja]

is it definitely 152? I would expect it to be https://github.com/jenkinsci/azure-ad-plugin/releases/tag/154.v12e17a5f9ea3

Which changed from the legacy ADAL to MSAL.

If that's the case then the above change probably won't help much. Now that there's more info I can probably have a go at checking this in detail.

To have it work now I would suggest using security groups instead.

[PrasannaShasthriDevOps]

My bad, its 155.v745ce80af7ea, as I mentioned in my first comment along with other plugins and their versions.

[Iridias]

I don't think Microsoft 365 groups should be allowed but given it worked before an option could be made for it to work.

That would be great.

I tested the issue by duplicating one of our groups and making it of type "Security" - that works, so its definitely the group-type, cause the issue. But as mentioned earlier: having to duplicate and manage multiple groups manually is not a good solution.

[PrasannaShasthriDevOps]

I think no new version got released for the PR #152

When I checked this step and click on details. continuous-integration/jenkins/incrementals Deployed to Incrementals.

I get this

{
  "errors" : [ {
    "status" : 404,
    "message" : "{\"error\":\"Item incrementals:org/jenkins-ci/plugins/azure-ad/174.v4193ef6db383 does not exist\"}"
  } ]
}

[timja]

@Shasthri it was released 11 hours ago: https://github.com/jenkinsci/azure-ad-plugin/releases/tag/174.vc2d906355813

I've checked and the plugin is available.

I've tested with mail enabled security and 365 groups, before https://github.com/jenkinsci/azure-ad-plugin/pull/152 there was an issue, but after all seems to work fine.

I've created https://github.com/jenkinsci/azure-ad-plugin/pull/153 to allow picking the groups in the people picker, but that won't affect people who already have that configuration

[PrasannaShasthriDevOps]

When that will be available here - https://plugins.jenkins.io/azure-ad/#releases for downloading ? Are you able to view latest version(v174) in the above link ?

[timja]

Why don't you just download it through Jenkins itself?

The plugin site is slower to update, but you can just update the url to the latest version if you want to download it yourself

Last published url is: https://updates.jenkins.io/download/plugins/azure-ad/173.v0a210fffb510/azure-ad.hpi

Just update the version number to the version you want.

[KalleOlaviNiemitalo]

Version 191.vfc8019068670 is already available from the Update Center within Jenkins, and from https://plugins.jenkins.io/azure-ad/#releases. Should this issue be closed?