Introduce new account app (to replace oauth app)

[kschiffer]

Summary

We currently have a lot of issues revolving around our OAuth flow, account switching, token management, respective branding, etc. This is an attempt to make all of these more actionable under a new account app that will take care of all of these concerns.

Why do we need this?

• Missing account switching functionality ⟶ #488
• Missing branding and context for the user ⟶ #730
• Missing session management ⟶ #1217
• User confusion around OAuth authentication flow ⟶ #265

What is already there? What do you see now?

• OAuth app with limited UI

What is missing? What do you want to see?

An app that acts as OAuth provider and fully featured and branded configuration UI around all account and authentication concerns

Environment

Web

How do you propose to implement this?

I think a big problem with the current implementation is the lacking clarity and context for the user. It is hard to recognize the OAuth app as an independent authentication entity. Reasons are:

• lack of branding and context
• very reduced and uninformative UI
• console acting as if it handles authentication itself ("Login" message instead of "Login with your {OAuth provider service name} account", skipping of authorization) To improve clarity, we need to make the app more visible, distinguishable and purposeful:
• Add configurable branding
• Flesh out functionality
  • Account switching
  • Session management
  • Authorizations management
  • Profile settings (change/forgot password, email, profile picture)

I've created three wireframes that give an idea of how I envision the account app:

See wireframes

   (notice the account switcher integrated in the user dropdown in the top right)

Can you do this yourself and submit a Pull Request?

Yes. @johanstokking @htdvisser, please let me know whether you think its good to continue.

[johanstokking]

References https://github.com/TheThingsIndustries/lorawan-stack/issues/1659

[kschiffer]

Regarding the whole login/logout confusion issue between console and oauth provider, here's my plan:

• We use a separate Account logo for the account app (instead of the TTS Logo) and use the same header component which we also use for the console (cc @pierreph)
• Instead of having two separate login screens for console and account app, we should tie the experience closer together, meaning
  • We will remove the console login screen /console/login
  • The console will automatically redirect to the account app login page /account/login, when it has no (valid) access token
  • Like we already do with the console login button, we will use a redirect query param to send the user back to the console after successful login
  • Logging out from the console UI will result in removing the stored access token, as well as logging out from account app. This way, users will always have to re-enter their credentials in order to log back into the console.
    • To be able to do this, we need a way to trigger logouts from cross-origin requests (since console and account app are not guaranteed to be on the same origin)
    • In v2, we're using a simple /users/logout route that will trigger a logout which can be referred to from anywhere
    • Would be good to look into ways to achieve the same thing while staying CSRF safe
  • The new flow will pretty much resemble the flow of the v2 console; we'll basically sacrifice the separation of console client and authorization server to improve the UX, which is OK since the console is sort of a special case client
  • We might need to revise this flow if we allow for different login procedures in the future

If I don't hear any complaints about this plan I'll start implementing this.

[johanstokking]

Sounds like a great plan. I agree we don't need explicit logout of Console and stay logged in via OAuth in the UX, although it will work that way underneath.

What do you need for cross-origin logout? Wouldn't it be a simple two-step approach; logout "page" to remove Console stored access token, then redirect to generic logout page where the Account app logs out? Or do you want this without redirection but post to a logout endpoint?

[htdvisser]

For logout, we can get some inspiration from OpenID Connect logout. Nice summary can be found here: https://medium.com/@robert.broeckelmann/openid-connect-logout-eccc73df758f

[kschiffer]

Sounds like a great plan. I agree we don't need explicit logout of Console and stay logged in via OAuth in the UX, although it will work that way underneath.

What do you need for cross-origin logout? Wouldn't it be a simple two-step approach; logout "page" to remove Console stored access token, then redirect to generic logout page where the Account app logs out? Or do you want this without redirection but post to a logout endpoint?

Two-step approach is OK. My concern was that the logout of the account app needs to be CSRF disabled, meaning that anyone with a logout link could lure someone into logging out from their account. This is also currently possible with the v2 stack (clicking here will log you out). I think for now this is acceptable but not exactly best practice.

[kschiffer]

I've done quite some work now for the new account app:

• Refactored components and containers that can be used by console and account app
• Implemented new designs (including responsiveness)
• Implemented profile settings, including profile picture
• Implemented authorizations management (view and revoke)
• Implemented session management (view and revoke)
• Resolving an issue that resulted in blank renders when using shared css module dependencies

I did this by hardcoding an access token, to be able to use the stack API. I'm currently unable to advance with a proper implementation, since I don't know what would be the best way to obtain an access token for the account app. Right now, the "oauth app" is just a SPA that connects to the auth endpoints of the oauth server (login, logout and other account related endpoint that do not need an access token). @htdvisser mentioned to me that our initial idea was to have the account app as separate oauth client, using the password grant type. I think we should open a discussion about this, since:

• depending on the source, the password grant type is discouraged, even for "official" and trusted clients
• it would necessitate another login screen (next to the login screen of the oauth provider)
• I'm wondering whether there is a way to get an access token from the oauth provider directly, without having to use another separate client, since we're already doing authentication inside the oauth app
• if we use the password grant for the account app, we should use it for the console as well

I already said to @htdvisser that the whole delegated authentication/authorization aspect of this is somewhat overwhelming to me and I don't feel knowledgeable enough to be responsible for such a security-sensitive matter.

What we should aim for is a flow that makes the authentication flow of our official clients feel like native authentication (see also my other comment), meaning that logins and logouts are global and there should be no distinction between the authenticated state of the clients and the authorization provider.

So, I'd need some help and input to go on with the account app. How can we coordinate this?

[johanstokking]

depending on the source, the password grant type is discouraged, even for "official" and trusted clients

Yes, password grant must not be used, see specification and reasoning here: https://tools.ietf.org/html/draft-ietf-oauth-security-topics-13#section-3.4

[kschiffer]

Currently blocked by #2148

[kschiffer]

So after being involved with this for a while now, I think I need some input.

The original idea of the account app was to replace the current oauth app and extend functionality towards profile and session management (see OP). The issue with this is that in order to implement such functionality, the app would need an access token, essentially meaning that the account app would be both an OAuth provider and client.

My idea was to introduce a new OAuth client (account) and frontend-wise, to blend the functionality with the current OAuth app. For the user, it would appear as though both OAuth provider (login, register, validate account, etc) and the account app client (profile settings, session management, authorization management) are one and the same application, while in the background, there is a technical separation between the OAuth provider and the client. On the backend, the account client would become a part of the oauth package. Frontend-wise we could also treat both things as the same app, using the same entry point (account.js). Likewise, the routing would reflect this blending, e.g.:

• /account/login for the oauth login [OAuth provider layer]
• /account/register for user registration [OAuth provider layer]
• /account/forgot-password for user registration [OAuth provider layer]
• /account/client/login/ttn-stack for the client login (authorization) [OAuth client layer]
• /account/client/oauth/callback to exchange the auth code [OAuth client layer]
• /account overview page of the oauth client [OAuth client layer]

Alternatively, we could keep both concerns separated, leaving the OAuth provider (/oauth via pkg/oauth) as it is and treating the account app (/account via pkg/account) as a completely separate thing, like we do with the console. The OAuth provider would then be only responsible for authentication matters, including authorization and user account registration, but would not have an authenticated view of its own as it currently has. Instead, after the login, it would redirect to the account app by default, which would then retrieve a token automatically.

I feel like the first solution might avoid some of the confusion which we currently have revolving around authentication and account management, but I cannot foresee whether it might introduce other complications. The second solution seems cleaner, but the question is how to communicate/brand the separation. E.g. should both be communicated as "Account app" still? Or should it be the The Things Stack Single Sign On and The Things Stack Account Application?

Please let me know your opinions.

[johanstokking]

Hmm, taking one step back, what it is the purpose of making the account app an OAuth client? What functionality via OAuth do we want in there?

[kschiffer]

• Profile settings (name, email, profile picture)
• Session management (review, revoke)
• Authorization management (review, revoke)
• (Potentially other TTES related meta settings in the future)

All of those necessitate an access token since that is the only way to authorize the respective RPCs.

[htdvisser]

We need a separation of the OAuth provider -- which could be external, for instance "The Things Network Community" -- and User management -- which is about managing fields of the User entity stored with The Things Stack, including the points @kschiffer wrote in the comment above.

Our /api/v3/* endpoints do not accept cookie auth, because they are CORS-enabled. They only work with API keys and Access Tokens.

[kschiffer]

• Profile settings (name, email, profile picture)
• Session management (review, revoke)
• Authorization management (review, revoke)
• (Potentially other TTES related meta settings in the future)

All of those necessitate an access token since that is the only way to authorize the respective RPCs.

We could also make these part of the console but it would mean that the console would mix concerns to some extend. I could see situations where it would be better to perform account management without the overhead of all the other console functionalities, but maybe that's me overengineering this 🤷‍♂️.

Generally, the question would be if we want to see the console only as a tool to manage network related matters. We can either open it up to be a general purpose management platform for all TTS related matters or we can be more atomic and stick with the separation and use different apps/clients for different concerns. It's a strategic question. I see cases for both.

[htdvisser]

I don't think the console (all consoles, because there's a console in each cluster) should have full rights on the user. Consoles shouldn't be used to manage authorized OAuth clients, sessions, primary email address, contact info. Having a dedicated User management app (the account app) is much better.

[kschiffer]

Indeed, good point. Then it comes back to the initial question.

My take is:

Blended approach:

• From a user POV less confusing (one location for all account related matters)
• Easier branding / communication
• Conforming with the original plan of the account app
• Frontend-wise, we don't need a new entry point and hence separate boilerplating
• Implementation becomes more complex/confusing (account app being oauth provider and client)

Seperated approach:

• Cleaner and more conventional implementation
• Configuration more flexible and clearer
• Introduces a new app next to the oauth app, which needs sensible branding and communication
• Longer builds (might be marginal, haven't tested) and more assets

I lean towards the blended approach.

[johanstokking]

• Profile settings (name, email, profile picture)
• Session management (review, revoke)
• Authorization management (review, revoke)
• (Potentially other TTES related meta settings in the future)

This is account app, right? Not OAuth?

• Introduces a new app next to the oauth app, which needs sensible branding and communication

What is the OAuth app for then still?

Can you list the features for the user of what the OAuth client adds to the account app, that we cannot build in the account app, potentially via new endpoints?

[kschiffer]

This is account app, right? Not OAuth?

Yes, this is account app. But the plan was that what we currently have as OAuth app becomes part of the account app.

What is the OAuth app for then still?

For authentication (as part of the OAuth flow), user registration, client authorization, password reset. Basically everything that has to do with authentication of users and authorization of clients.

Can you list the features for the user of what the OAuth client adds to the account app, that we cannot build in the account app, potentially via new endpoints?

The issue is more the other way around, we cannot simply extend our current OAuth app with the functionalities we want for the account app, since we'd need to get an access token. It's a bit of a chicken and egg problem. If there was a different means of authorization for the account app functionalities (sessions, authorizations, etc), e.g. via the session cookie that the OAuth app has, then it would be different. But from what I see this is unfeasible.

[johanstokking]

Is the OAuth app a stand-alone thing? Is it implementing the redirect flows for OAuth clients? Or is it an OAuth client itself?

---

Emphasis mine:

My idea was to introduce a new OAuth client (account) and frontend-wise, to blend the functionality with the current OAuth app. For the user, it would appear as though both OAuth provider (login, register, validate account, etc) and the account app client (profile settings, session management, authorization management) are one and the same application, while in the background, there is a technical separation between the OAuth provider and the client. On the backend, the account client would become a part of the oauth package.

I see here;

• OAuth client
• current OAuth app
• OAuth provider
• the account app client
• the OAuth provider
• the client
• the backend
• the account client
• the oauth package

I'm trying to understand the proposal, but I don't fully understand what is what still.

So what are the features that we want?

1. Create account
2. Forget password
3. Login
4. Change password
5. View and edit profile
6. Manage sessions
7. OAuth stuff
   1. Consent screen (OAuth authorization code flow)
   2. Manage OAuth authorizations

I guess that 1-6 can be implemented without touching OAuth at all, if we go with session authentication. That is possible, right?

[kschiffer]

Yeah sorry. We have a number of unfixed terms here that make it quite difficult to explain things. At least it gives a good insight into the complexity of the issue 😅.

So let's take the status quo:

OAuth provider The entity that provides authorization (and in our case also authentication) according to the OAuth 2.0 specification. In our case that's the pkg/oauth package, which in turn uses the OAuth app (see below) to implement the necessary user interface to obtain user information (render login, registration, authorization views, etc.).

OAuth app This is the react web application on the frontend. Our frontend code consists of different app (with different entry points oauth.js / console.js) which share common parts such as react components and utilities

OAuth client A registered client that uses OAuth for authentication and authorization, like the console or CLI.

oauth package The Go package that is responsible for implementing the OAuth provider. That's pkg/oauth.

So you see, there are three or four different layers at play here. The problem is that if we keep the way we do authorization at the moment, we'd need to have something that plays all roles outlined above at the same time.

I guess that 1-6 can be implemented without touching OAuth at all, if we go with session authentication. That is possible, right?

Yes. 1-4 are already possible without oauth / access token. 5 and 6 currently need an access token. 7. ii is not yet implemented. If 5 and 6 would be possible without access token then we would indeed not need another oauth client to do this. Still we need to consider then that every functionality we might want to add to the account app in the future must then not use access token as sole means of authorization.

[johanstokking]

If 5 and 6 would be possible without access token then we would indeed not need another oauth client to do this. Still we need to consider then that every functionality we might want to add to the account app in the future must then not use access token as sole means of authorization.

Right. This makes a lot of sense to me. This is also better groundwork for "sudo mode", where users can do things only in the account app, after re-entering your password for example. So yes, there might be some overlap between what OAuth clients and the account app can do, but that overlap doesn't seem big to me. I see value in keeping things dedicated to the account app.

[kschiffer]

Alright, sounds good. @htdvisser Do you have any objections? And if not, can you point me to the relative files/packages for authorization, because I suppose you won't have time to work on this any time soon (?).

[htdvisser]

As I commented before, our /api/v3/* endpoints do not accept cookie auth, because they are CORS-enabled. If you want to start accepting cookie auth, you would need to take a good look at our CORS headers, because we really don't want an attacker to make cookie-authenticated cross-origin requests to these endpoints.

---

Our API is all gRPC, and auth is done with the Authorization header, which is propagated to gRPC as request metadata. Implementation of session auth could look like this:

• Add a SessionToken type to pkg/auth/auth.go
• Add a secret part to the Session model, similar to what we do with API keys and Access Tokens.
• Add a middleware that extracts the session ID and session secret from the cookie and forwards it into the Authorization header as a SessionToken
• Add SessionToken as a case to the switch tokenType { in pkg/identityserver/entity_access.go

[kschiffer]

One thing to consider here is that this also means that the account app and the v3 API need to be served from the same origin. Otherwise, cookie authentication wouldn't work as the auth context comprises different origins. Not sure how acceptable that is. At least in practice, we seem to be using the same origins in our deployments.

[johanstokking]

Yes, they will be on the same origin.

[johanstokking]

@kschiffer please let us know how we can unblock the progress here.

[kschiffer]

I wrote a summary here: https://github.com/TheThingsNetwork/lorawan-stack/pull/2850#issuecomment-657497193

[johanstokking]

OK, let's continue that conversation here.

@kschiffer After working on this for a bit:

• The frontend part of the oauth app would basically consist of the authorization screen only

Yes

• I think it's very annoying having to retain the is.oauth.ui.*  configs just for that [...]
• I'm also a little bit unsure about the exact restrictions of how we can alter the stack configuration without breaking the CC
• I'm a bit out of luck with finding a viable solution here and really need some input

We can't break configuration in V3, but we can introduce new configuration, deprecate the old configuration, use the old configuration as fallback, and file a TODO issue labeled bump/major to remove the old config.

So my suggestion would be to introduce new configuration that would be a complete replacement and enables the new account app. For backward compatibility, it should work without the user specifying this new configuration, i.e. rely on sane default values and old config via is.oauth.ui.*.

[johanstokking]

@kschiffer what is the status here?

[kschiffer]

Currently rebasing my work on the new scaffold, which is ongoing here: #3453

Next up is rebasing, fixing and PRing the authenticated views and adjusting the current designs to the new branding.

[johanstokking]

@kschiffer what is the status here?

[kschiffer]

The Account App has been introduced in 3.11. Currently still missing are:

• Account switching #488 
• Session management 
• Authorizations management (iirc this also includes adding API endpoints)

[johanstokking]

OK. https://github.com/TheThingsNetwork/lorawan-stack/issues/488 is already closed it seems.

This has been a big issue. Can you file one or two new issues for the above to replace this one, so it can be closed?