400-style errors when signing in via OAuth2 with Nextcloud

[Eisfunke]

I use CodiMD in a Docker container using the current master branch. I configured OAuth2 with my Nextcloud following the directions.

The set environment variables concerning OAuth2 are:

CMD_OAUTH2_CLIENT_ID=**snip**
CMD_OAUTH2_CLIENT_SECRET=**snip**
CMD_OAUTH2_AUTHORIZATION_URL=https://**my cloud**/apps/oauth2/authorize
CMD_OAUTH2_TOKEN_URL=https://**my cloud**/apps/oauth2/api/v1/token
CMD_OAUTH2_USER_PROFILE_URL=https://**my cloud**/ocs/v2.php/cloud/user?format=json
CMD_OAUTH2_USER_PROFILE_USERNAME_ATTR=ocs.data.id
CMD_OAUTH2_USER_PROFILE_DISPLAY_NAME_ATTR=ocs.data.display-name
CMD_OAUTH2_USER_PROFILE_EMAIL_ATTR=ocs.data.email

Now, every time when I try to sign in using OAuth, I get forwarded to my Nextcloud just fine. I sign in, grant the request, Nextcloud forwards me to https://**my codimd**/auth/oauth2/callback?state=&code=**snip**.

That's when CodiMD crashes with an uncaught exception (which is why I get an 502 Bad Gateway error in the browser). The console output is:

app_1_54315b635fcd | 2018-11-14T02:30:47.572Z - error: uncaughtException: passport.InternalOAuthError is not a constructor date=Wed Nov 14 2018 02:30:47 GMT+0000 (UTC), pid=1, uid=10000, gid=65533, cwd=/codimd, execPath=/usr/local/bin/node, version=v8.12.0, argv=[/usr/local/bin/node, /codimd/app.js], rss=76795904, heapTotal=48164864, heapUsed=44126440, external=131771, loadavg=[0.34326171875, 0.37548828125, 0.4228515625], uptime=4913, trace=[column=19, file=/codimd/lib/web/auth/oauth2/index.js, function=null, line=70, method=null, native=false, column=9, file=/codimd/node_modules/oauth/lib/oauth2.js, function=passBackControl, line=132, method=null, native=false, column=7, file=/codimd/node_modules/oauth/lib/oauth2.js, function=null, line=157, method=null, native=false, column=20, file=events.js, function=emitNone, line=111, method=null, native=false, column=7, file=events.js, function=IncomingMessage.emit, line=208, method=emit, native=false, column=12, file=_stream_readable.js, function=endReadableNT, line=1064, method=null, native=false, column=11, file=internal/process/next_tick.js, function=_combinedTickCallback, line=139, method=null, native=false, column=9, file=internal/process/next_tick.js, function=process._tickCallback, line=181, method=_tickCallback, native=false], stack=[TypeError: passport.InternalOAuthError is not a constructor,     at /codimd/lib/web/auth/oauth2/index.js:70:19,     at passBackControl (/codimd/node_modules/oauth/lib/oauth2.js:132:9),     at IncomingMessage.<anonymous> (/codimd/node_modules/oauth/lib/oauth2.js:157:7),     at emitNone (events.js:111:20),     at IncomingMessage.emit (events.js:208:7),     at endReadableNT (_stream_readable.js:1064:12),     at _combinedTickCallback (internal/process/next_tick.js:139:11),     at process._tickCallback (internal/process/next_tick.js:181:9)]
app_1_54315b635fcd | 2018-11-14T02:30:47.573Z - error: An uncaught exception has occured.
app_1_54315b635fcd | 2018-11-14T02:30:47.573Z - error:  message=passport.InternalOAuthError is not a constructor, stack=TypeError: passport.InternalOAuthError is not a constructor
app_1_54315b635fcd |     at /codimd/lib/web/auth/oauth2/index.js:70:19
app_1_54315b635fcd |     at passBackControl (/codimd/node_modules/oauth/lib/oauth2.js:132:9)
app_1_54315b635fcd |     at IncomingMessage.<anonymous> (/codimd/node_modules/oauth/lib/oauth2.js:157:7)
app_1_54315b635fcd |     at emitNone (events.js:111:20)
app_1_54315b635fcd |     at IncomingMessage.emit (events.js:208:7)
app_1_54315b635fcd |     at endReadableNT (_stream_readable.js:1064:12)
app_1_54315b635fcd |     at _combinedTickCallback (internal/process/next_tick.js:139:11)
app_1_54315b635fcd |     at process._tickCallback (internal/process/next_tick.js:181:9)
app_1_54315b635fcd | 2018-11-14T02:30:47.574Z - error: Process will exit now.

Any ideas what the problem is? Thanks in advance!

[ccoenen]

I had an error in a similar place, I can't recall if it was exactly the same. My CodiMD and Oauth services run on docker containers each, and one wasn't able to make a network call to the other. I was missing DNS setup, if I remember correctly.

I confirmed this by logging on to each one and pinging the other by its full domain name (not any shorthands like they are used in docker).

[ccoenen]

but the described error is a little weird, in that this line should not fail. Because this line 70 fails, we're actually losing the err that occurred while fetching the profile in line 66.

https://github.com/hackmdio/codimd/blob/df2a2e6c6c38a59b4cbabd95487dd6e9a35160e3/lib/web/auth/oauth2/index.js#L66-L70

Either way: the error in question occurs when the "profile" information would be obtained by CodiMD (CodiMD asks your OAuth service for things like username, so it can be displayed as author info).

[ccoenen]

I believe i can make a quick fix for the described code error which masks your root cause.

[ccoenen]

Could you try out these changes and let me know how it goes?

The Error will not be gone, but instead of 502 you should get a readable error message about what actually happened.

(edit: I messed up one thing with the import which should be fixed now)

[Eisfunke]

Thank you! When I use the bug/oauth2internalerror branch from your repo CodiMD crashes on startup though :/

app_1_54315b635fcd | 2018-11-14T11:32:55.618Z - error: uncaughtException: Class extends value undefined is not a constructor or null date=Wed Nov 14 2018 11:32:55 GMT+0000 (UTC), pid=1, uid=10000, gid=65533, cwd=/codimd, execPath=/usr/local/bin/node, version=v8.12.0, argv=[/usr/local/bin/node, /codimd/app.js], rss=95064064, heapTotal=75005952, heapUsed=43977120, external=73502, loadavg=[1.1220703125, 1.5009765625, 1.15576171875], uptime=37441, trace=[column=36, file=/codimd/lib/web/auth/oauth2/index.js, function=null, line=11, method=null, native=false, column=30, file=module.js, function=Module._compile, line=653, method=_compile, native=false, column=10, file=module.js, function=Module._extensions..js, line=664, method=.js, native=false, column=32, file=module.js, function=Module.load, line=566, method=load, native=false, column=12, file=module.js, function=tryModuleLoad, line=506, method=null, native=false, column=3, file=module.js, function=Module._load, line=498, method=_load, native=false, column=17, file=module.js, function=Module.require, line=597, method=require, native=false, column=18, file=internal/module.js, function=require, line=11, method=null, native=false, column=43, file=/codimd/lib/web/auth/index.js, function=null, line=46, method=null, native=false, column=30, file=module.js, function=Module._compile, line=653, method=_compile, native=false, column=10, file=module.js, function=Module._extensions..js, line=664, method=.js, native=false, column=32, file=module.js, function=Module.load, line=566, method=load, native=false, column=12, file=module.js, function=tryModuleLoad, line=506, method=null, native=false, column=3, file=module.js, function=Module._load, line=498, method=_load, native=false, column=17, file=module.js, function=Module.require, line=597, method=require, native=false, column=18, file=internal/module.js, function=require, line=11, method=null, native=false], stack=[TypeError: Class extends value undefined is not a constructor or null, at Object.<anonymous> (/codimd/lib/web/auth/oauth2/index.js:11:36), at Module._compile (module.js:653:30), at Object.Module._extensions..js (module.js:664:10), at Module.load (module.js:566:32), at tryModuleLoad (module.js:506:12), at Function.Module._load (module.js:498:3), at Module.require (module.js:597:17), at require (internal/module.js:11:18), at Object.<anonymous> (/codimd/lib/web/auth/index.js:46:43), at Module._compile (module.js:653:30), at Object.Module._extensions..js (module.js:664:10), at Module.load (module.js:566:32), at tryModuleLoad (module.js:506:12), at Function.Module._load (module.js:498:3), at Module.require (module.js:597:17), at require (internal/module.js:11:18)]

Meanwhile, I'm still trying to find out why CodiMD can't fetch the profile. The network setup is not the problem, the URL of my cloud resolves correctly an can be connected to.

When I try wget **my cloud**/ocs/v2.php/cloud/user?format=json from the CodiMD container I get HTTP error 401 Unauthorized though.

[ccoenen]

I'm sorry, I realized my mistake a few seconds before, I just pushed an updated version to the server, under the same branch.

[ccoenen]

Thanks for the 401 mention. Then it is at least a different error than I had. We can get into that, once the InternalOAuthError works as intended. If you wish, you can hop onto matrix (A chat tool), to discuss this more interactively.

[Eisfunke]

Yeah, it works now. When signing in, the server doesn't crash anymore. In the browser I get a page with »Internal Server Error« and the following appears in the log:

app_1_54315b635fcd | InternalOAuthError: Failed to fetch user profile
app_1_54315b635fcd |     at /codimd/lib/web/auth/oauth2/index.js:70:19
app_1_54315b635fcd |     at passBackControl (/codimd/node_modules/oauth/lib/oauth2.js:132:9)
app_1_54315b635fcd |     at IncomingMessage.<anonymous> (/codimd/node_modules/oauth/lib/oauth2.js:157:7)
app_1_54315b635fcd |     at emitNone (events.js:111:20)
app_1_54315b635fcd |     at IncomingMessage.emit (events.js:208:7)
app_1_54315b635fcd |     at endReadableNT (_stream_readable.js:1064:12)
app_1_54315b635fcd |     at _combinedTickCallback (internal/process/next_tick.js:139:11)
app_1_54315b635fcd |     at process._tickCallback (internal/process/next_tick.js:181:9)

Actually, I would join the matrix room, but I currently have trouble with my matrix server, I can't join rooms, and I didn't get around to try fixing it, sorry :joy:

[ccoenen]

:-D

Alright, we're getting somewhere. Sadly, this still somewhat obsuceres the error. Perhaps @SISheogorath has an idea where such an error might end up. Maybe there's a different place where passport errors end up?

[Eisfunke]

So... some information I gathered:

The CMD_OAUTH2_USER_PROFILE_URL=https://**my cloud**/ocs/v2.php/cloud/user?format=json URL is part of Nextcloud's OCS API, it has nothing directly to do with OAuth.

When I call curl -i 'https://**my cloud**/ocs/v2.php/cloud/user?format=json', I get a 401 Not Authorized error with the message »current user is not logged in«.

This is expected behavior as (of course) this information isn't public and one needs to sign in to retrieve it. See here.

So, following the information from the link I generated an app token as password and executed curl -i 'https://**user**:**pass**@**my cloud**/ocs/v2.php/cloud/user?format=json'. Now, instead of the 401, I get an 412 error with the message »CSRF check failed«.

According to this Nextcloud issue that's because I need to send the OCS-APIRequest: true header. And indeed, calling curl -i 'https://**user**:**pass**@**my cloud**/ocs/v2.php/cloud/user?format=json' -H "OCS-APIRequest: true" correctly returns JSON with my profile data.

Is maybe one of these the problem (missing authentication or header) when CodiMD tries to retrieve profile data?

[SISheogorath]

I just checked a bit around nextcloud and it seems like they fixed it earlier this year and now broke it again -.-

We have to check with upstream what's their plan. The current way doesn't allow any generic setup.

I guess you run Nextcloud 14, right? As far as I can say it worked on Nextcloud 13 and broke on my own Nextcloud instance with version 14 (as I just checked). But I'm also not willing to write a workaround just for Nextcloud because once we start that, we'll write more and more exceptions and special handlings over time, which will probably become unmaintainable.

So I guess upstream issue it is.

[Eisfunke]

Yes, I'm using Nextcloud 14. What exactly did they break though?

[ccoenen]

this is odd. I'm running nc 14 and codi 1.2.1 side by side with oauth2.

[SISheogorath]

When I interpret it correctly, I would look into this: https://github.com/nextcloud/server/issues/5694

According to this when you use a token (which oAuth2 should do), it should work without CSRF error.

Mhm but somehow I just realized I took the idea that the problem is caused by a CSRF error, which doesn't have to be the case .-.

[ccoenen]

I changed the title to reflect that a tiny part of the problem has been resolved.

[Eisfunke]

If the CSRF check is indeed the problem, and NC isn't fixing it, maybe adding a CMD_OAUTH2_USER_PROFILE_HEADER (or something like that) config option could be added, allowing the user to set headers to be send with the profile request? I don't know if that counts as Nextcloud-specific workaround :D

[ccoenen]

Then I still don't get why it "works for me"?

[Eisfunke]

@ccoenen What happens if you call curl -i 'https://**user**:**pass**@**my cloud**/ocs/v2.php/cloud/user?format=json' on your Nextcloud with an app token? Do you get the CSRF error?

[ccoenen]

here's the final two lines from the apache log file from nextcloud:

[15/Nov/2018:22:55:34 +0000] "POST /apps/oauth2/api/v1/token HTTP/1.1" 200 1598 "-" "Node-oauth"
[15/Nov/2018:22:55:34 +0000] "GET /ocs/v2.php/cloud/user?format=json HTTP/1.1" 200 1658 "-" "Node-oauth"

I'll try curling next.

[ccoenen]

I can curl successfully with this:

curl -H 'Authorization: Bearer yBA---some-token---zWo6' 'https://<username>:<pwd>@<server>/ocs/v2.php/cloud/user?format=json'

and it will return the json I expected. (with -I, of course it would return headers)

[ccoenen]

I got the Authorization: Bearer .... header by sniffing the unencrypted traffic between my reverse-proxy and my application server. It was set automatically in the oauth library when doing the oauth2-login. I just took it and applied it to curl. It's the only header I took.

[ccoenen]

To re-iterate:

curl does need to set the Authorization: Bearer ... header. curl does NOT need to set a OCS-APIRequest header. codi (more precisely passport-oauth2) should do this correctly on its own. this has been tested with codi 1.2.1 (docker deployment, no changes) and nextcloud 14.0.3.0 (docker deployment, no changes, no extra apps activated)

curl as suggested by me above with just the username/password/url will return 412 precondition failed / CSRF-Token missing. The curl test was just proposed to see if the dns/routing/networking between the two machines was fine (which it is either way, even if the returned value is an error-HTTP-result). In case you didn't continue testing Codi itself because curl was still failing, please retry with codi itself, now that we know that networking shouldn't be the problem.

[SISheogorath]

WOOOW This is an awesome issue!

But I figured it out! I had to add some debug logging in our code base as well as the depending oauth2 library. But it worked out. Before I saw in the nextcloud log that it responded with an HTTP code 303.

So as mentioned, added some debugging code. First in our oAuth module:

https://github.com/hackmdio/codimd/blob/f9aa001ee78e604415c2a6e82f65701a1f07d3c4/lib/web/auth/oauth2/index.js#L69-L71

I added some lines:

      console.log(err)
      console.log(res)

Which turned out, res was empty as well as body. And error was quite short, since it only contained the response code 303.

So I needed to extend the error message itself a bit. Since we use JS and in JS we use callbacks, let's climb up. Where do we pass the callback to?

https://github.com/hackmdio/codimd/blob/f9aa001ee78e604415c2a6e82f65701a1f07d3c4/lib/web/auth/oauth2/index.js#L66-L84

The this._oauth2.get()-function. So good so far. Time to look into that. As it turns out, this function is from passport-oauth2:

https://github.com/jaredhanson/passport-oauth2/blob/a9480960be04ac5312f9eae2a6d06d0e8d0e5dda/lib/strategy.js#L88-L92

But of course this is only the remapping. So up the tree:

https://github.com/ciaranj/node-oauth/blob/a7f8a1e21c362eb4ed2039431fb9ac2ae749f26a/lib/oauth2.js#L219-L228

Of course our actual call isn't in here either, so we search the _request()-function: https://github.com/ciaranj/node-oauth/blob/a7f8a1e21c362eb4ed2039431fb9ac2ae749f26a/lib/oauth2.js#L70-L121

Which turns out to call the _executeRequest()-function: https://github.com/ciaranj/node-oauth/blob/master/lib/oauth2.js#L123-L137

So we finally found our function. The critical part is, besides the condition, the error it pushes: https://github.com/ciaranj/node-oauth/blob/master/lib/oauth2.js#L132

callback({ statusCode: response.statusCode, data: result });

Extending the error with the actual response then solved the missing information problem:

callback({ statusCode: response.statusCode, data: result, response: response });

As it turns out, we were redirected to the following URL:

'Location: /login/selectchallenge?redirect_url=/ocs/v2.php/cloud/user%3Fformat%3Djson',

And when you ever setup multiple 2FA providers for your Nextcloud account, you know that this is the view to decide which one you want to use for your current login.

So all in all: This error is caused by the multi-factor authentication implemented in Nextcloud. I'm not sure if it appears when you use a single 2FA provider for your account (like TOTP) but when you use multiple, it seems to be the case and causing problems.

I'm not sure if htaccess.RewriteBase = true is part of the problem or not, but it explains why it works for some people and for others it doesn't. I guess it's still an upstream bug as I'm not aware of what we could do about that.

Hope this helps and explains a bit how to debug such a problem, if someone ever want to go for such a horror. It would probably be easer to use a debugger here but I was in a live environment. Lesson for today: Don't strip away too much information with an error. Always provide all information you have to your callbacks.

[ccoenen]

this is some nice detectivework!

[Eisfunke]

Nice, thank you for the commitment!

I'm using only TOTP, so the problem appears with only a single 2FA provider enabled.

[SISheogorath]

@Eisfunke do you want to take care of creating an upstream bug and linking it here? Would be awesome :)

[Eisfunke]

@Eisfunke do you want to take care of creating an upstream bug and linking it here? Would be awesome :)

Sure! See here.

[Eisfunke]

The nextcloud issue should be fixed now. As soon as it lands in my Nextcloud version I'll test it with CodiMD and close this (of course anybody else could do it as well).

[SISheogorath]

Good to hear, thanks for your work!

[ccoenen]

Releases to look for:

• 13.0.9 13.0.8 https://nextcloud.com/changelog/#13-0-8
• 14.0.5 14.0.4 https://nextcloud.com/changelog/#14-0-4
• 15 (any)

[dampfklon]

Nextcloud 14.0.4 released on 22.11 contains the fix

[ccoenen]

nice! It used to be 14.0.5, but has been prioritized, apparently.

• https://github.com/nextcloud/server/pull/12545#event-1981992333
• https://github.com/nextcloud/server/pull/12545#event-1977011550

[Eisfunke]

I just checked: With Nextcloud 14.0.4 sign-in via OAuth2 works just fine. Thanks to all! :+1:

[ccoenen]

Thanks for checking back :-)