[RELEASE]: Multi tenancy UI dashboard + account linking fixes + telemetry change to include websiteDomain + email verification claim change + extending jwks lifetime

[rishabhpoddar]

📅 Checklist

Release order

• [x] Python sdk jwks changes
• [x] Core
• [x] Dashbaord
• [x] Node - with backend sdk testing for FDI 3.0
• [x] add tests for fdi 1.17, 2.0 for backend sdk testing repo - containing apiversion interceptor, tests for includeInNonPublicTenantsByDefault and email verification claims
• [x] golang release
• [x] docs updates
  • [x] update oidc urls in snippets and text around them (https://github.com/supertokens/docs/pull/819)
  • [x] type fix for golang snippets
• [x] python release
• [x] docs type check with python
• [x] add tests for fdi 1.18, 1.19 for backend sdk testing repo - containing apiversion interceptor, tests for includeInNonPublicTenantsByDefault and email verification claims
• [x] node release - with backend tests parallelised and compatible with 1.18 and 1.19
• [x] Docs
  • [x] https://supertokens.slack.com/archives/D03CNFWTP99/p1721141369573889
  • [x] general changes related to dashboard
• [x] move dashboard tests from node sdk to backend-sdk-testing
• [x] update curl commands of release script of core to be as per what is at the bottom of the file below

🔶 Staging

Dev Tag

• supertokens-core:X.Y
  • [x] core
    • [x] recompile with right version of openjdk
    • [x] check CDI, plugin interface list
    • [x] Add migration script for psql / mysql
    • [x] Make sure no memory leak
    • [x] Make sure new telemtry API is pushed to prod
  • [x] plugin-interface
    • [x] check plugin interface list
  • [x] mysql-plugin
    • [x] check plugin interface list
    • [x] Add migration script for mysql
  • [x] postgresql-plugin
    • [x] check plugin interface list
    • [x] Add migration script for psql
  • [x] mongodb-plugin
    • [x] check plugin interface list
• [x] supertokens-node:X.Y
  • [x] check CDI, FDI list
  • [x] email veirfication claim logic change
  • [x] jwks refresh lifetime to 4 hours
  • [x] Make sure all example app pr checks are passing
  • [x] SAML fix
  • [x] https://supertokens.slack.com/archives/D03CNFWTP99/p1721461342761369
• [x] supertokens-golang:X.Y
  • [x] check CDI, FDI list
  • [x] apiversion telemetry change
  • [x] boxyhq additional config change
  • [x] network interceptor
  • [x] test server for backend-sdk-testing
  • [x] jwks url logic change
  • [x] email veirfication claim logic change
  • [x] jwks refresh lifetime to 4 hours
  • [ ] multi tenancy ui
  • [ ] account linking fixes
  • [x] add includeInNonPublicTenantsByDefault in ProviderInput
• [x] supertokens-python:X.Y
  • [x] check CDI, FDI list
  • [x] apiversion telemetry change
  • [x] boxyhq additional config change
  • [x] network interceptor
  • [x] test server for backend-sdk-testing
  • [x] jwks url logic change
  • [x] email veirfication claim logic change
  • [x] jwks refresh lifetime to 4 hours
  • [ ] multi tenancy ui
  • [ ] account linking fixes
  • [x] add includeInNonPublicTenantsByDefault in ProviderInput
  • [x] supertokens-auth-react:X.Y
  • [x] check FDI list
  • [x] check web-js interface version
  • [x] Updated dependencies to use supertokens-web-js in package.json from npm registry
  • [x] supertokens-web-js:X.Y
  • [x] check FDI list
  • [x] check web-js interface version
  • [x] email verification claim change
• [ ] supertokens-dashboard
  • [x] Tested all items mentioned in this? https://github.com/supertokens/dashboard/blob/master/.github/PULL_REQUEST_TEMPLATE.md
  • [ ] Some of the images in the PR are huge
  • [ ] Change twitter image to X image
  • [ ] change api-key in header to be Authorization bearer token
• [ ] Test day with team. Get people in the team to read the docs and implement something with the new feature.

Others

• [x] Example apps in create-supertokens-app CLI
  • [ ] Examples apps in supertokens-auth-react. Update try.supertokens and rerun the pr checklist
  • [ ] Examples apps in supertokens-node. Update try.supertokens and rerun the pr checklist
  • [ ] next.js:canary
  • [ ] SuperTokens Jackson SAML example update
  • [ ] Merge the with-multi-tenancy-ui branch from rishabhpoddar/multi-tenancy-mfa to the main branch

📚 Documentation (test site)

• [ ] All recipe main documentation update
• [x] Migration in changelog for node sdk
• [ ] Change all curl snippets to the core to use authorization bearer token instead of api-key
• [x] Code type checking versions are pointing to X.Y
  • [x] jsEnv
  • [x] goEnv
  • [x] pythonEnv
• [x] Update table schema in mysql / postgresql section for self hosted
• [ ] website changes (test.supertokens.io)
  • [ ] tenant management UI in the feature page
• [ ] ~https://supertokens.com/docs/thirdpartyemailpassword/pre-built-ui/setup/backend~
  • [ ] ~We should make backend setup first in the docs, and then frontend setup.~
  • [ ] ~In backend setup, we should only tell users to setup init, middleware, error handler -> visit dashboard login and configure tenant. This means we will have to enable all recipes by default in backend, which is OK i think.~
  • [ ] ~In frontend setup, we should always add useDynamicLoginMethod: true, and always init all recipes, so users just have to call init.~
• [ ] ~https://supertokens.com/docs/mfa/backend-setup~
  • [ ] ~For MFA, we should tell users to config via dashboard only, and for complex stuff, they can do via code.~
• [ ] In dashboard tab, we should remove that it's only for node sdk

  🔥 Production

💻 NPM and core release

• core
  • [x] supertokens-core:X.Y
  • [x] plugin-interface
  • [x] mysql-plugin
  • [x] postgresql-plugin
  • [x] mongodb-plugin
  • Docker update
    • [x] MySQL
      • [x] check if new env cnofigs need to be added
    • [x] Postgres
      • [x] check if new env cnofigs need to be added
    • [x] MongoDB
      • [x] check if new env cnofigs need to be added
  • [x] try.supertokens.io

    docker rm try-supertokens -f
    docker rmi supertokens/supertokens-postgresql:<VERSION>
    nano ~/try-supertokens/start_container.sh (update version tag)
    ~/try-supertokens/start_container.sh

  • [x] Update SaaS instances to use the latest docker images.
  • [x] Change checklist in contributing guide for which tables to pick when migrating data from dev to prod instance.
  • [x] Update table schema in mysql / postgresql section for self hosted in docs.
  • [x] Update logic for core to core migration for new saas architecture:
    • [x] transfer of master database information
    • [x] deletion of master database information related to the CUD being transferred
• [x] supertokens-node:X.Y
• [x] supertokens-golang:X.Y
• [x] supertokens-python:X.Y
• [x] supertokens-web-js:X.Y
• [x] supertokens-auth-react:X.Y
• [x] supertokens-dashboard

📚 Documentation

• [ ] Pushed to production
• [ ] Post message on discord about new update
• [x] Update core-driver-interface-repo
• [x] Updated swaggerhub CDI spec
• [x] Updated dashboard spec on swaggerhub
• [x] Update dashboard spec
• [x] Update internal contributing guide to move from previous core version to the latest one
• Auto generate release note on github:
  • [x] supertokens-core
  • [x] supertokens-plugin-interface
  • [x] supertokens-mysql-plugin
  • [x] supertokens-postgresql-plugin
  • [x] supertokens-mongodb-plugin
  • [x] supertokens-node
  • [x] supertokens-golang
  • [x] supertokens-python
  • [x] supertokens-web-js
  • [x] supertokens-auth-react
  • [x] supertokens-dashboard

Contents of running try.supertokens.com script:

docker run -d \
    --restart=always \
    --name try-supertokens \
    --label name=try-supertokens \
    --label type=session-service \
    --label mode=production \
    --log-driver=awslogs --log-opt awslogs-region=ap-south-1 --log-opt awslogs-group=try-supertokens --log-opt awslogs-stream=try-supertokens \
    -e DISABLE_TELEMETRY=true \
    --publish 9999:3567 \
    supertokens/supertokens-postgresql:6.0

sleep 7

curl --location --request POST 'https://try.supertokens.com/recipe/dashboard/user' \
--header 'rid: dashboard' \
--header 'api-key: <YOUR-API-KEY>' \
--header 'Content-Type: application/json' \
--data-raw '{"email": "rishabh@supertokens.com","password": "abcd1234"}'

curl --location --request POST 'https://try.supertokens.com/recipe/dashboard/user' \
--header 'rid: dashboard' \
--header 'api-key: <YOUR-API-KEY>' \
--header 'Content-Type: application/json' \
--data-raw '{"email": "demo@supertokens.com","password": "abcd1234"}'

curl --location --request PUT 'https://try.supertokens.com/recipe/multitenancy/tenant/v2' \
--header 'Content-Type: application/json' \
--data-raw '{
    "tenantId": "tenant1",
    "firstFactors": ["emailpassword", "thirdparty"]
}'

curl --location --request PUT 'https://try.supertokens.com/tenant1/recipe/multitenancy/config/thirdparty' \
--header 'Content-Type: application/json' \
--data-raw '{
  "config": {
    "thirdPartyId": "google-workspaces",
    "name": "Google Workspaces",
    "clients": [
      {
        "clientId": "1060725074195-kmeum4crr01uirfl2op9kd5acmi9jutn.apps.googleusercontent.com",
        "clientSecret": "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",
        "additionalConfig": {
            "hd": "*"
        }
      }
    ]
  }
}'

curl --location --request PUT 'https://try.supertokens.com/recipe/multitenancy/tenant/v2' \
--header 'Content-Type: application/json' \
--data-raw '{
    "tenantId": "tenant2",
    "firstFactors": ["emailpassword"]
}'

curl --location --request PUT 'https://try.supertokens.com/recipe/multitenancy/tenant/v2' \
--header 'Content-Type: application/json' \
--data-raw '{
    "tenantId": "tenant3",
    "firstFactors": ["otp-email", "otp-phone", "link-phone", "link-email", "thirdparty"]
}'

curl --location --request PUT 'https://try.supertokens.com/tenant3/recipe/multitenancy/config/thirdparty' \
--header 'Content-Type: application/json' \
--data-raw '{
  "config": {
    "thirdPartyId": "github",
    "name": "GitHub",
    "clients": [
      {
        "clientId": "467101b197249757c71f",
        "clientSecret": "e97051221f4b6426e8fe8d51486396703012f5bd"
      }
    ]
  }
}'

[rishabhpoddar]

Dashboard issues

• [x] OIDC discovery endpoint in provider config deosn't have the well-known path and all

• [x] If i add a provider config via API (like google workspaces), and then I visit that on the dashboard, it does not show the hd in additional config (NOW)

• [x] supertokens_max_cdi_version should not show up in core config on dashboard (NOW). This will be done via a new annotation in the core like @HideFromDashboard which will prevent it from showing up for self hosted or saas.

• [ ] If I enable third party for a tenant, and then i click on cancel in the popup that it keeps third party enabled. Is that ok? If so, then once third party is enabled, we can directly show the third party selection ui. If the user clicks on cancel there, show a warning on the UI saying that at least one provider must be selected, otherwise an error will be thrown (if no other first factor is enabled).

• [ ] for self hosted users, the config in public tenant are not modifiable at all on the dashboard - pretty bad experience.

• [x] For saas users, what config will they be able to modify in non root CUD, public app, public tenant? (NOW)

• [ ] If i add a a custom provider, and do not specify any oidc endpoint nor the authorization urls etc, it allows me to save it anyway. We should change the flow to accept third party id and name before, and then based on that, the form should have prefilled values for OIDC or other endpoints, if those are configured in the core or static for this third party id. If they are not configured in the core or static, the then user must fill in OIDC OR (authorization, user info, token exchange).

• [x] Name should be compulsory for custom SAML and custom OAuth (NOW)

• [x] When adding SAML provider, the name should not be SAML, but it should give the user an option to choose from a list of names that we have support for (see this), including SAML. Show a drop down (NOW).

• [x] The API keys config should be hidden from the dashboard? It's a security risk, no? (NOW)

• [x] For SAML, need to allow users to upload xml and stuff on our dashboard

• [x] When modifying an existing SAML provider, the urls for authorization and all have undefined instead of the previously provided url (NOW)

• [ ] In provider listing (the ones that are already added), we need to separate out the oauth and saml providers in the UI

• [x] If i have init passwordless recipe with only email, and then i clikc on otp-phone in mfa, it says "please init passwordless recipe". This is a wrong error message. It should instead say, "please change contact method of passwordless.init on the backend" (in case passwordless is initilaised, else the message right now is fine)

• [x] First factor is otp-email, second factor is totp and otp-phone. After logging in with first factor, the /auth/mfa screen shows nothing. No error on console either. (NOW)

• [ ] Write down steps for adding new social provider.

• [x] Drop down for saml name should only be shown if the name is not coming from the backend or we create a new provider. For example, in the curl command, if i set the name to "Mock SAML", the name should show as that in the provider config (NOW)

• [ ] we should show an info that changes in the dashboard will not reflect in the frontend if useDynamicLoginMethods is not set to true (for pre-built UI)

  • [ ] this should be updated in the docs as well

• [x] bug in delete user UI (see screenshot)

  

• [ ] In the provider config dashboard, if I have overriden the userInfo function, it says "Custom override". Change it to "Cannot edit this because you have provided a custom override"

• [ ] Need to tell users on tenant management UI that frontend's useDynamicLoginMethod: true in our frontend sdk.

• [ ] Issue in search tag while switching tenants (the search results doesn't take search tags into consideration)

• [ ] In provider config UI, need to add placeholders for all fields, so users know what the format is of the various fields. Especially for things like apple private key.

• [ ] One thing also for dashboard is that if someone tries to create a new tenant, we have to tell them it’s a paid feature. Also, if they have not added multi tenancy, it should show them that error properly and not something went wrong.

• [ ] Under the "Secondary Factors" section in the tenant config UI, explain that enabling will require all users in this tenant to setup one of those factors . If you want to do conditional MFA setup and requirement, then override the getMFARequirementForAuth function in the backend sdk config instead and keep these switched off here.

• [x] with API key, manual save is not working

• [x] change manual to 'via boxy api'

• [ ] in docs, update boxy curl command to use auto-generated values for tenant and product

• [x] check for non 200 in getThirdPartyInfo api

[rishabhpoddar]

Thoughts on 20th july, 2024

I also thought about how we can change the docs and flow so that by default, users dont have to init stuff on the frontend (pre built ui) and can go to the dashboard directly to start configuring the tenants and login methods. This is possible if we:

• On the frontend, init all recipes and pre built uis
• Set usesDynamicLoginMethods: true by default
• On the backend, init all recipes by default

The issue with the above is that it breaks the flow of setting up:

• Users would first need to add the backend init, middleware, error handler
• Connect it to their core (and not try supertokens)
• create dashboard credentials
• configure login methods
• setup frontend

So they get real feedback really late in the setup phase. As opposed to now, where in the first step is setup frontend, and then get to see the pre built UI immediately. Which is why we didn't go ahead with these changes