OIDC login to Gitea 500 error `UserSignIn: oauth2: server response missing access_token` despite it exists

[NexZhu]

Description

Hi, I'm trying to use kanidm as OIDC authentication source for Gitea, it used to work for the previous version of Gitea (I forgot the version), I've upgraded Gitea to the latest v1.20.2 since, and now after redirecting back to /usr/oauth2/Kanidm/callback, the page shows 500 and Gitea's error log says:

2023/08/09 07:56:50 ...eb/routing/logger.go:102:func1() [I] router: completed GET /user/oauth2/Kanidm for 172.16.40.213:57572, 307 Temporary Redirect in 1.5ms @ auth/oauth.go:849(auth.SignInOAuth)

2023/08/09 07:37:27 ...rs/web/auth/oauth.go:923:SignInOAuthCallback() [E] UserSignIn: oauth2: server response missing access_token

2023/08/09 07:37:27 ...eb/routing/logger.go:102:func1() [I] router: completed GET /user/oauth2/Kanidm/callback?state=69fbc6a2-6e08-40e4-aa33-67e9ef226803&code=gAAAAABk00I2HwG8QH_0P40nva3-sqyXzled4Dres5sj1gYXTI5G1U0ToEpWRFRp2a5s8Z9BdGN07skh_4ix0NOZ8hv58nvhlFyRZ58zS_Sb7B31jNG_Cv1rv8ixzr9yKS2ozDQr1864XnniWJMo_TMwTwxTHzoA1cUNvN7DC9Y6Z56_5ay527AslKZwH5qMkCIS1Kac_sagGhK5CpMcDnvJAl0QDoLf8SdkF6D30SrclN6VleDww5aierC3dgUE0Z7-wLiIhiC2NhK8uhWWcTUrbx3LyruC6_XPVjGuDkr2e0O0ueLQkk_ypIf7WCCZ3TVaqJ4GE50l_Sv6bU2all6G-rpaz1NmlsVmYF8bH1w4C99xkdCWcXyTC1AC8cQ4Vgh7J3y5_1_5aeeiviUK3wE6Bto3CJMZBEXfKarY4w1jQf1QIYh1qeUr4Os4Rvidp8_iaDqxvdrRCmV2-XVex_qAXdF0ADrGj7nYkONPYshgilUE5ybRXclSPCAptdcwOVq5IxWRxGt5Tw36LSRr_vg3W592ptlh-JKD9V9Lte9GDC2VZk0gD5zPH6FVpCGgNKqbuEmMpp8z4t3kXEEcshZ7nZO0-YWJZRlTN8f756cdrzOxeb-MvDcu0ylrHw9c0TQwYmgiEL-754Y9xKAZjoZJQBxwRCRNdBGQPCxYcYY8chXHiROMEc1A8ZxyEIlVaTDb3J9m0qVwIJdQsUnH_Nj_xt8-R0P5gZVN6Nuw8PHGxkRWhzwbqag%3D for 172.16.40.213:35320, 500 Internal Server Error in 354.9ms @ auth/oauth.go:886(auth.SignInOAuthCallback)

Which is strange, because on the Kanidm side there's no error and I've log the HTTP response with a debug reverse proxy in the middle, and access_token clearly exists:

kanidm-debug-proxy-1  | POST /oauth2/token HTTP/1.1                                                                                    [34/1939]
kanidm-debug-proxy-1  | Host: idm.fusiongalaxy.cn
kanidm-debug-proxy-1  | Accept-Encoding: gzip
kanidm-debug-proxy-1  | Authorization: Basic Z2l0ZWE6ZHJDZzBmSFV4VDlVMlRadVlTYTRBRWNIazVnMzhTV2YzZXlydHAzYnFGSE5razRn
kanidm-debug-proxy-1  | Content-Length: 881
kanidm-debug-proxy-1  | Content-Type: application/x-www-form-urlencoded
kanidm-debug-proxy-1  | User-Agent: Go-http-client/2.0
kanidm-debug-proxy-1  | X-Forwarded-For: 8.142.30.60
kanidm-debug-proxy-1  | X-Forwarded-Host: idm.fusiongalaxy.cn
kanidm-debug-proxy-1  | X-Forwarded-Proto: https
kanidm-debug-proxy-1  | 
kanidm                | 41560c8a-9343-4d4f-96d9-1564a01c2d96 INFO     request [ 4.67ms | 2.80% / 100.00% ]
kanidm                | 41560c8a-9343-4d4f-96d9-1564a01c2d96 INFO     ┕━ handle_oauth2_token_exchange [ 4.54ms | 50.67% / 97.20% ]
kanidm                | 41560c8a-9343-4d4f-96d9-1564a01c2d96 INFO        ┕━ commit [ 2.17ms | 46.53% ]
kanidm-debug-proxy-1  | code=gAAAAABk0z6nnSEaZSB5z0ZWOH5KN1mlpb2tj8CZiBKYdr4geDQ18P0Ql-HMXX7VA0nMOb5RRQe1agJo6Mvz9j35nuTvjB8SzakMi5nztBmPi8O3veY
vI14BnoJ0_LNqvkMntdpFgsOIHFcLZZ2nP0myLkmI3EqQeZQPiCoA7eOxAjV0AxF6g7OR1iUHeO50C7kqwvKl3E8vzhHbTNteu3eafJA3-YfClr07FbUuXab2JLOXxUEgkDxj_BkIrr2p3QD
d38U6aa2HQq7V-i_UfVstjKd51dZhCS08AecWdZaU5cif9bNzsLUEIbOOE-n6BBioq1w8xcozwif91xhjuFDW-Y612907jHrj4tb_PUjErz32SKl-G3IWN3fyF9dHi7I8KsP2n3Kc7ew9lN3
c3PWseSni_A1f402bI_5IxAmeyAz-QNq-BKpyWkBIQ0ZjrxbGuqh1WqrocZU4I2eeKb3ynvVA-G5_U4wanCbHVOsB3N6R5M0fQEzkP7pNGQDWXBiaes1dOqVl_y8OQI3ZXSJITlbwYQjrli1
h44pvykkiQsJYXYXdleFhkxRpNIqA5WaM_sJXt37ZfDv0pdckW5aFnyR1BfUBBsqgZDwCjCiSvB93NvCXFpjP7g5wDjhp9wOZrMb98GOQzoTCruv_lEpfm-2dhw55lkSTZeZek0SPZSJrO0c
hXOj036fzyJfvrJMnzLB9Ts-7N5fsqREYaKvmuomBBNOLPXJIlBKKYXVnCcVzNg_Qzco%3D&grant_type=authorization_code&redirect_uri=https%3A%2F%2Fgitea.redacted.com%2Fuser%2Foauth2%2FKanidm%2FcallbackHTTP/1.1 200 OK
kanidm-debug-proxy-1  | Content-Length: 2150
kanidm-debug-proxy-1  | Access-Control-Allow-Origin: *
kanidm-debug-proxy-1  | Cache-Control: no-store no-cache max-age=0
kanidm-debug-proxy-1  | Content-Security-Policy: base-uri 'self' https:; default-src 'self'; form-action 'self' https:; frame-ancestors 'none';
img-src 'self' data:; script-src 'self' 'unsafe-eval' 'sha384-MrcW6ZMFYlzcLA8Nl+NtUVF0sA7MsXsP1UyJoMp4YLEuNSfAP+JcXn/tWtIaxVXM' 'sha384-Zao7ExRX
VZOJobzS/uMp0P1jtJz3TTqJU4nYXkdmsjpiVD+/wcwCyX7FGqRIqvIz'; worker-src 'none';
kanidm-debug-proxy-1  | Date: Wed, 09 Aug 2023 07:22:17 GMT
kanidm-debug-proxy-1  | Permissions-Policy: fullscreen=(), geolocation=()
kanidm-debug-proxy-1  | Pragma: no-cache
kanidm-debug-proxy-1  | Referrer-Policy: no-referrer-when-downgrade
kanidm-debug-proxy-1  | Strict-Transport-Security: max-age=86400
kanidm-debug-proxy-1  | X-Content-Type-Options: nosniff
kanidm-debug-proxy-1  | X-Kanidm-Opid: 41560c8a-9343-4d4f-96d9-1564a01c2d96
kanidm-debug-proxy-1  | X-Kanidm-Version: 1.1.0-beta.13
kanidm-debug-proxy-1  | 
kanidm-debug-proxy-1  | {"access_token":"gAAAAABk0z6pmCVYFLfRWBaYqMNkzJYoiQYasCiPi6EKxjVpDKPfww0HWS45irMQlrh3byAJC-QldnyGeSX-vnMl4tEKHEGMHGefNcM
2mhtuoYgCEES4Kcmz2iZE6TdkZEP9azgE3tRe8IJ1ZZy12h-Ag7mVdMGkuovgm4i8JtMsROtZD_UDk5kFrSCie8YIGb1BYBu_dQmLyK153zy3rRv1YpQ4J2nfBw9YEosWdcnmMqPIotm9LnK
Zdr35sBD9cAoMawVrCYeDwODvwtc0NhxXK4t6Lz6GNGQXEWDTD1wwbq0GKQYd-92uO5jexI1QmA2RtUrm2U8y-5gGt-yYL-cvRuSFNqUUIBOSjGcQnrC_jM2j79V0P4-HPQuv_DPBEDcxk3V
b6XOp0cSvmYSbq7523DneVnu9NsyltsMVRdmAY1yn1iBYor_Z3YV037pBha3T8tHVWzl4X6gvYj62a7W_kDdoHeW9zQ==","token_type":"bearer","expires_in":900,"refresh_t
oken":"gAAAAABk0z6pYsD_2ZHgBXD1OYKHqE-JHAJ1jhEatlKKuXel351UHaIYR4RbO-cn4TzXF82jX1zKFnQgl3Rb-_w5tX6jvAvXhi_gkpDFyriSR2xCq4fdsDsuea5AEfTctqGO_8Q2G
kn3l7MnqFaNISxemv3YjvvyLsFvYVCzaSn8z2ahwI13Php4iL5vqHBL10NhTNHWk9RkQCsI3wdBumlNJzV8kHdUzxey7bPjzq-A8KZYoTd_HsAun5_0WGrCcTlhon07y2UAIQmLxs5S95Lql
DzPytn3Z8WkuQkK8mq8SvBhHgWoV2KOTF77zAaA2k5z60Nn8cfM-tDan0-vW2a8CuMhxXDKVX00xCTqJTXaYUL3KpOyXLeajH_Ip7lLhHsVCYeLh6kfBNJ_ktvy6fOgADWNcC5DP48zfaNhb
efH0i6_lkmzdv35-ArgdGsp2xVSSOrPtiI6","scope":"email groups openid profile","id_token":"eyJhbGciOiJFUzI1NiIsImtpZCI6IjlhZTQ1OGI1NGQyYTQ1YjY1NDY3Y
zY4YzkxYjVjZjdkNzA4MDBlOWMwMTJhY2M3NTQxYjY0NmY0OGEyODA1N2EiLCJ0eXAiOiJKV1QifQ.eyJpc3MiOiJodHRwczovL2lkbS5mdXNpb25nYWxheHkuY24vb2F1dGgyL29wZW5pZC
9naXRlYSIsInN1YiI6IjM0MzdlNWExLWRiMGUtNGVlYy1hZTg3LWRjNGE2NzdjOWMzMiIsImF1ZCI6ImdpdGVhIiwiZXhwIjoxNjkxNTY2NjM3LCJuYmYiOjE2OTE1NjU3MzcsImlhdCI6MT
Y5MTU2NTczNywiYXV0aF90aW1lIjpudWxsLCJhenAiOiJnaXRlYSIsIm5hbWUiOiLmnLHmmZPml7siLCJwcmVmZXJyZWRfdXNlcm5hbWUiOiJ6aHV4aWFvbWluIiwiZW1haWwiOiJ6aHV4aW
FvbWluQGZ1c2lvbmdhbGF4eS5jbiIsImVtYWlsX3ZlcmlmaWVkIjp0cnVlLCJzY29wZXMiOlsiZW1haWwiLCJncm91cHMiLCJvcGVuaWQiLCJwcm9maWxlIl0sImdyb3VwcyI6WyIwMDAwMD
AwMC0wMDAwLTAwMDAtMDAwMC0wMDAwMDAwMDAwMzUiLCIwMDAwMDAwMC0wMDAwLTAwMDAtMDAwMC0wMDAwMDAwMDAwMzYiLCI0NGU2MDEwOC0yZjcxLTRmZDEtYTkzZS03YTY1N2MyNmI0NW
UiLCI4MWZhNzIxOS00MmFjLTQyZDktOWM1NS03YjVkY2VjYzQ2YzgiLCJhYTEyNTA5NC1iZGZlLTRhYTUtOWZiZS0xZjc0Y2RjMWExNzYiLCI2M2I5NjRmYi00ZDQ5LTQ4NDItOTIxZS1jNz
VkNTA5ODkxZDAiLCIzNDM3ZTVhMS1kYjBlLTRlZWMtYWU4Ny1kYzRhNjc3YzljMzIiXX0.FPlruBHtRMXL1ikI90UX8HigIJxcr4Ad5axhPmzXTgtQ1pl4soAkdwbyvYfpGF5HZxQIOBOhHH
Nk6kZeSq1wpw"}

I have not renamed the OAuth2 application. It used to work with older version of Gitea, so I think the OIDC provider is fine, could this be a bug in the latest version?

Gitea Version

1.20.2

Can you reproduce the bug on the Gitea demo site?

No

Log Gist

No response

Screenshots

No response

Git Version

No response

Operating System

K8s

How are you running Gitea?

K8s with the official Helm chart

Database

PostgreSQL

[NexZhu]

Help needed🙏

[lunny]

Which provider did you use in fact?

[techknowlogick]

@lunny https://kanidm.com/

[NexZhu]

Yes, with Kanidm it had worked once, but I've upgraded Gitea version since. I also logged the response from Authelia which is working, and compared with the Kanidm one, which has one more field refresh_token, I don't think it could be the cause though.

[pfalzsocial]

I can reproduce that (found this issue by googling for the very same problem). Following...

[pfalzsocial]

According to this writeup, it rather seems to be an issue with kanidm... https://ashhhleyyy.dev/blog/2023-02-05-from-keycloak-to-kanidm

[NexZhu]

@pfalzsocial According to the blog post you posted, it should work with the latest Kanidm, however it's failing for me with the latest Kanidm 1.1.0-beta.13. Also Gitea complaining server response missing access_token when it actually exists, is at least not an accurate error message, we still need some help from the Gitea team to find out what's the real cause of the 500 error.

[NexZhu]

@lunny @techknowlogick Any advice how I can debug further?

[yaakov-h]

@NexZhu See the discussion here, this seems to be a fixed issue on Kanidm's side: https://github.com/kanidm/kanidm/discussions/2058

Though I do agree that Gitea's error message could do with refinement to indicate the actual problem.

[Firstyear]

The issue is that Kanidm was incorrectly returning a response without a correct content type header (for anyone who wants to know the answer without having to dig through all the issues/code). This is resolved in our devel images aka rc.14. Sorry about the issues you had here @NexZhu :(

[williamdes]

I have the same issue, on my first run to connect with GitHub I have web/auth/oauth.go:937:SignInOAuthCallback() [E] UserSignIn: could not find a matching session for this request. Then I go back to login and try once more, and it works. Reproduced on different users.

This bug has been there for years, I only report it now

[williamdes]

@wxiaoguang should I open a new issue for GitHub ? I also have DISABLE_REGISTRATION: true

[wxiaoguang]

I have the same issue, on my first run to connect with GitHub I have web/auth/oauth.go:937:SignInOAuthCallback() [E] UserSignIn: could not find a matching session for this request. Then I go back to login and try once more, and it works. Reproduced on different users.

This bug has been there for years, I only report it now

If it is a different problem, feel free to open a new issue with a reproducible setup (ideally by a docker compose with detailed steps), then if some people have time, they would take a look.

[deadbeatz]

I believe we are running into the same issue with OIDC on Gitea 1.21.7. In our case, we are authenticating against an Azure B2C tenant with a custom user flow. After we authenticate in the tenant, it redirects to the GItea callback and gives us error 500.

I stood up a dummy authentik docker to test this Gitea OIDC with and it works fine in the same instance. Something specific to the Azure B2C is causing the problem.

2024/03/06 02:46:35 ...rs/web/auth/oauth.go:937:SignInOAuthCallback() [E] UserSignIn: oauth2: server response missing access_token

2024/03/06 02:46:35 ...eb/routing/logger.go:102:func1() [I] router: completed GET /user/oauth2/azureb2c/callback?state=767d0f86-c4ef-4012-a7f3-8049791c792f&code=eyJraWQiOiJjcGltY29yZV8wOTI1MjAxNSIsInZlciI6IjEuMCIsInppcCI6IkRlZmxhdGUiLCJzZXIiOiIxLjAifQ..OUaJNMESA7glOvJ3.g_59tvq-jd_9dW-O5YEtm5lcD-K4Pne6BETJMd7cXVB0_uSLYVAEsgAr7e-CmJ0ImfS2hkTVf_sGqyj2WunK2CugpZGNgWG2WZcjkrAi2G8dVy-rOrRgO3UKwN35uI-LZTJ8F0SX8xdmv6ML_iPzD50bZzDxLtMFDU3u5Uo4BVLXqFxHyvYf4kZ2hNkT1Fs1YgitHUhGbmhN9HMEsA2cEplKYVrsAWl6KpH0-mDbIL6ENuir78Cg1-0ya23DXZpTO9vbBtOGhcHWUIBbXOrgBUaQQM0kdRo0voOqBVRY6uSZMLkDKLkBAHijffBkH4eA6TjnxBuJCvMiExpLniJriiTkHjE2wyjzG0KNNEosBZhxzdiw5P1ve3XaLyJTjj3__6viD6TJmXt3XPL1-k4_vXabsvfolXHwvL77Ra15nx0OS9I8Ibxjl9EjmI1a4rlh04lEG0PbxglhXj9w0C1MKAq46XEN.f4nXicml_eHJ8BKwMOYp6w for 10.244.2.142:47874, 500 Internal Server Error in 642.6ms @ auth/oauth.go:886(auth.SignInOAuthCallback)

Setup in Gitea authentication: