search

redwoodjs / redwood

The App Framework for Startups
https://redwoodjs.com
MIT License
17.34k stars 996 forks source link

RFC RW Multi Tenant Support #5821

Open pantheredeye opened 2 years ago

pantheredeye commented 2 years ago

Multitenancy has been brought up several times over the past few weeks. @dthyresson requested that an RFC be started.

Goals:

Current Status:

The current approach leaning towards developing a 'How-To' and/or to extend the RW test project.

@dthyresson:

For me, teams and auth is an app and business and data modeling and access/auth problem that isn't necessarily "cookie cutter". I agree that a great first start is a how to — and then see maybe what needs to be in the framework The data model and query can basically be the same as https://canary.blitzjs.com/docs/multitenancy for an example implementation The missing piece is the auth and filtering on context but really that same info can be stored on the currentUser just like roles are It’s pretty much just storing the org and memberships in the currentUser and then some nice joins

@Brett-D (@brettdoyle44):

Agree with you there… the more I think about it the more I believe a short tutorial on extending redwood for multi-tenancy makes the most sense. The auth section could have three parts showing how to implement with Clerk orgId structure, Supabase/GoTrue structure, and then dbAuth structure to give people some options there

If anyone wants to checkout my progress on trying to make a Saas Starter using Redwood (disclaimer: I am not a professional developer haha). https://github.com/brettdoyle44/redwoodjs-saas-starter

Notes

See references below in Community Approaches/Mentions.

Database Architecture?

DB Options:

So far, from posts I have collected, most approaches seem to use the single database, single schema (inferred) approach.

Edit: See this comparison between different models (Single Schema/Multi Schema/Multi DB). The single db/single schema (row level security) approach feels like it aligns with Redwood more than the other options. This approach is best suited where the data structure between tenants is the same, which fits with the prisma schema. This approach is also noted in several sources as the easiest approach to scale.

Schema/DB Model

General DB model structures seem to revolve around the bullet train/blitz model. It connects Organizations and users via the membership table.

Auth

@dthyresson Discord Mention: Clerk offers 'Organizations' as part of their product. @KrisCoulson: RFC: Extend useAuth with custom functions #2431

Routing

Subdomain option: customer1.domain.com and customer2.domain.com URL Path option: teamstream.gg/org1, teamstream.gg/org2/eventCode

Organizational Switching/Session/Context Handling

If users are part of multiple organizations, would RW need to help handle switching between orgs during a user session?

Other Mentions:

Generators, Directives, Services, Helpers Prisma Middleware Docs Writeup or Starter Template

Community Approaches/Mentions:

Single DB & Blitz Model

From this question by Milecia which kicked off a multi-day discussion on multi-tenancy:

KrisCoulson — 07/16/2021 @Milecia i am using redwood with multi tenancy as simon said are you trying to use multiple dbs? Or just one db. I am handling it with one db. I implemented it myself but after the fact found this guide on blitz which is pretty much how I am handling it. https://blitzjs.com/docs/multitenancy

viperfx — 07/17/2021 @KrisCoulson do you support switching orgs in your UI? I am curious to see how you are handling sessions and how your redwood app handles the "current" orgID for the user.

KrisCoulson — 07/18/2021 @viperfx currently their is not really a switching orgs UI but a user will be able to be apart of multiple orgs. But the way my app is set up I just have the orgs directly in the url. teamstream.gg/org1, teamstream.gg/org2/eventCode I modeled it similarly to github. github.com/redwoodjs

Then in your services and no with secured services I run this 

export const requireMembership = ({ role, org } = {}) => {
  requireAuth({ role })

  if (typeof org === 'undefined') {
    throw new ForbiddenError("You don't have access to do that.")
  }
  if (
    typeof org !== 'undefined' &&
    typeof context.currentUser.memberships[org] === 'undefined'
  ) {
    throw new ForbiddenError("You don't have access to do that.")
  }

  return context.currentUser.memberships[org]
}

From @Tobbe

Tobbe — 07/19/2021 I also do multi-tenancy, and I also built it before reading that blitz post. My table names are obviously different, but other than that my setup is very similar as well. My top level entity (my "organization") is a Team. What's a little bit different is that I then have two kinds of what Blitz calls "memberships", I have Players and Coaches. So a User can be a Player in different Teams, but also a Coach in different teams. Theoretically a User can even be a playing coach, so both a Coach and a Player in the same Team. But I also haven't built any profile-switcher yet. But when I do it'll probably be an option on the settings page. Or possibly an icon with a drop-down menu in the top right corner. But 99% of our users will most likely just be one user, connected to one player or coach, connected to one team. So it's not very high-prio right now to get it done. Or / page is our landing page with login and register buttons. After someone logs in I check if they're a coach, if so I redirect to /coach otherwise I redirect to /player. I imagine I'll have to add a profile picker later when I do the front-end support for switching profiles.

Clerk Use:

BSKnuckles — 06/15/2022 I'm doing this currently and am mixing a bit of Clerk's Organizations features with multi-tenancy data like Blitz demonstrates. When using Clerk you can include the organization memberships in your currentUser if you want to allow users to toggle between different orgs and only query data related to the one they are "active" in.

Clerk definitely helps make the auth side of this model easier. Then it's just a matter of adding some organizationId or userId columns to your data models and filtering returned data by those columns in your services (or maybe a custom directive that takes care of that filtering. Something I need to look into).

Using Subdomains:

@viperfx uses subdomains & routing to separate tenancy. @MaxLynam also wants to take this approach. Subdomains and custom domains.

I am looking to configure and setup subdomains and eventually custom domains for customer-controlled subdomains? I am using netlify for hosting, and Cloudflare for DNS. The app is currently hosted at app.domain.com I am looking to do support for customer1.domain.com and customer2.domain.com

@MaxLynam: We are at the final stages of a :scream: eight year R&D project … huge backend data-AI brain platform, for which we need to build out a WebApp for, which will have many organisations with their own accounts - and would be looking to have their own sub-domain for their organisational account (and possibly for their child-orgs as well).

Viper posts their routing solution in the thread (linked above).

Honorable mention from discord on subdomains:

biel cruz — 11/22/2021 Thanks! I'm still learning how to make basic authorization and CASL on redwood but I think the path is clear now. The thing is almost always when people talk about multitenancy is about domains (org1.example.com org2.example.com), I don't see many sources about teams and organizations in admin tools. I think that this is curious as almost all new saas apps are organization based (figma, notion, gitbook etc) and permission based (imagine github api access) not role based.

Directives/Services:

@simoncrypta mentions possibly using directives for multitenant logic and then @realStandal & @dthyresson have a deeper discussion three posts down in: V0.37 Release-Candidate is Available: Feedback Wanted.

Forum question about using a separate database with multitenancy; @ajoslin103 brings up directives: Separate database multi tenant architecture.

Generators/Docs:

Another thread touching on using a generator or creating some docs for multi-tenancy: SDLs for explicit m-to-n relations

SaaS Multi Tenant Starter

From 'Anyone have plans to make a Multi-Tenant SaaS starter on top of Redwood?':

https://www.1upblitz.com/ https://usegravity.app/

@Brett-D: Ideally out-of-the-box Teams/Workspace schema (not super hard). Authentication flow that supports the model. Invitation engine to invite team members to workspace. Subscription engine (via Stripe prob). Maybe some more advanced authorization to help secure APIs based on role.

@dthyresson Thoughts:

@dthyresson outlines some thoughts in: Org and team auth rules:

The user can create multiple organizations/teams and you can create permissions or roles based on the organization.

I don't think these is any prefab premade solution for this. But I would always start out with a matrix. A Matrix of roles, teams, organizations to see who does and needs what. Once you get the picture of your needs, maybe it's more simple than you think. Or you can implement parts of it incrementally. Most important, though, even before enforcing the rules is to capture the structure of your team and organization then you also have data vs app access to consider.

To start, build an org structure and then practice writing sql; queries to do the filtering access you need. Then you can work backward to capture the roles/permissions and how they may be associated with the team or org or user. The real tricky part is precedence. if a team permission says no but your user says yes, is it yes or no? It's all based on your app's business rules

Finally, from this thread, @dthyresson's latest thoughts:

At its most fundamental, one needs a data and access model as well as some way of stored memberships on a user. This can be done in many ways and RedwoodJS as yet doesn’t offer an opinion as to what that should look like. Other third party auth providers like Clerk have a way of storing those memberships. At its core though, each service needs to enforce some where clause query or join to ensure that a user cannot fetch or update data it should not. Because the currentUser is in context there could be some helpers to extract out memberships and then help form those query criteria on each Prisma call. Or perhaps Prisma middleware could also apply that criteria or otherwise limit and protect what the user can do

Can see they are just suggesting something similar to what I said: data access model, store memberships in session/context, and query filtering. But maybe some helper that makes that membership check is valuable.

Or:

... at some point orgs and teams could just be a how to. Not necessarily need a setup — but maybe the framework needs helpers or a separate package Maybe the package like @redwood/multi-tenancy has helpers and setup commands to run to generate the schema

Working Examples:

@orta

A working example of this is: https://github.com/orta/redwood-jwt-2phase-auth#jwt-2-phase-auth-in-redwood

I have Account which you could think of as a team account, and then sets of members 'User' attached to that team. When you log in it creates an auth token linking both your account and your user, making it easy to grab those during API requests.

Resources

https://blitzjs.com/docs/multitenancy https://blog.bullettrain.co/teams-should-be-an-mvp-feature/

White Paper on Performance: Multi-tenancy Design Patterns in SaaS Applications: A Performance Evaluation Case Study I question case study reports sometimes, but this outlines the common patterns.

Row Level Security (RLS)

Edits

06/28/2022: Added Working Examples section; Added RLS Resources & new DT thoughts 06/29/2022: Added Current Status section 07/02/2022: @dthyresson Added AWS Data isolation with RLS; Multi-tenant data isolation with PostgreSQL Row Level Security 07/07/2022: Updated with Brett-D GitHub template

orta commented 2 years ago

A working example of this is: https://github.com/orta/redwood-jwt-2phase-auth#jwt-2-phase-auth-in-redwood I have Account which you could think of as a team account, and then sets of members 'User' attached to that team. When you log in it creates an auth token linking both your account and your user, making it easy to grab those during API requests.

michaelmcnees commented 2 years ago

I don't have anything to add to the requirements, but I started working on my own version of a SaaS starter with Redwood. I'm happy to share that or help contribute towards an "official" starter.

brettdoyle44 commented 2 years ago

First, just want to say this is a great write up! I am currently building the core of a multi-tenant app with Redwood, Supabase Auth, and Mysql (local with docker) / Planet Scale for production as the DB. Happy to share anything I currently have. I used the Blitz/Bullet Train approach of having a membership table to create the relationship between user and team. I chose Supabase Auth because it is super straight forward and has a really attractive pricing model that ends up being much cheaper than others like Auth0 or Clerk. The other nice thing about it is it's easy to create a project for dev and one for prod to keep the data clean for testing.

pantheredeye commented 2 years ago

Updated with current status section based on discord chat around this topic.

pantheredeye commented 2 years ago

@dthyresson I missed your update in the contributor's meetup regarding this topic. Do you mind recapping the gist of it? Also, adding the below just to get my thoughts on the topic down, even if it is not the path we are really moving toward. I keep remembering your notes that multi-tenancy is app-specific and should be mapped out before implementing. Thus, I am hesitant to keep adding more to this topic. I also realize I don't know a ton about how this would integrate with RW overall. Anyway, FWIW:

I appreciate the 'simplicity' and 'drop-in' nature of prisma-multi-tenant's approach. The cool thing about this is that it seems like the prisma schema does not have to change like the blitz model. This makes it attractive to quickly develop an initial app using a single tenant design, and then be able move to a multi-tenant solution without a lot of data modeling changes.

It is a simple idea: 1) create a separate db for each tenant, 2) manage the tenants in a 'management' database and 3) then extract parameters (headers, auth/user data, url paths, etc) to connect the prisma client to the correct db.

From the Redwood example (linked below), here is the updated api/src/functions/graphql.js (They also replace the export from the db.js file):

import { multiTenant } from 'src/lib/db'

export const handler = createGraphQLHandler({
  schema: makeMergedSchema({
    schemas,
    services: makeServices({ services }),
  }),
  context: async ({ event }) => ({
    // The name can come from anywhere (headers, token, ...)
    db: await multiTenant
      .get('dev') // or 'my_tenant_A' or anything
      .catch(console.error),
  }),
})

The package author has created a cli to help manage the different tenants (listing, adding, deleting, etc), opening prisma studio to a specific tenant, and even running prisma migration operations on single tenants or all tenants at once. This approach uses different db url's for each client, so I think you could use one postgres db with schemas per client since they will have different urls denoting the schema.

Note: The package seems to be outdated at this point. Several github issues are created noting that it is not working with the current prisma cli. Still stands strong as an overall good implementation of this approach.

Redwood implementation here. Blogpost implementation write-up here.

Tobbe commented 2 years ago

@pantheredeye I hadn't heard of prisma-multi-tenant before. Thanks for bringing it to my attention 🙂 From what I can tell though, you need can't create any tenants on the fly from within your app. Is that correct?

For my use case it wouldn't work to manually create tenants. When a new user signs up they get to choose if they want to join an existing tenant or create a new one.

So my vote still goes to a "Single DB, Single Schema" How-To

Tobbe commented 2 years ago

Below I'm going to describe my setup. It's more complicated than the most basic multi-tenant setup. If we were to write a How-To guide, should we make it about a more complicated system, like mine, or the most basic system we can think of?

Complicated: ➕ We show a more real-life like example ➕ If you know how to set this up, you can easily build something simpler if that's all you need ➖ Much more difficult to implement and understand ➖ Takes longer to write

Simple: ➕ Easier for more people to understand ➕ Quicker to write up ➖ Users might feel a little left out in the cold when they go build the (probably more complicated) system they need

What should we focus on?

This is part of my current DB schema

image

When you first sign up a new user is created. You choose if you're a Player or Coach. If you're a Player you join a Team If you're a Coach you select if you want to join as a second coach in an existing Team or if you want to create a new Team

A Team has many Players and can have many Coaches A Player can only play in one Team, but a User can have multiple Players, all in different Teams A Coach can only Coach one team, but a User can have multiple Coaches, all in different Teams A User can have both Players and Coaches

Each Team has multiple PerformancePhases, which are periods of time (e.g. three months) where the Team is supposed to focus on getting better in a few different "development areas"

So when a User is logged in, to know if this user can access a given developmentAreaId I first need to look up what PerformancePhase that development area belongs to. When I have the PerformancePhase I know what teamId (i.e. tenant) I'm trying to access. Now I need to look at the User and see if he/she is currently logged in as a Player or Coach, and what Team that Player or Coach is playing in. When I know all this I can allow or deny access to the development area

pantheredeye commented 2 years ago

Hey @Tobbe,

Here are some of my thoughts:

What should we focus on?

Multi-tenancy can be a big topic; IMO starting simple is best. Your outline is a good start. If this follows the pattern of the other How To's then it will be a blend of rw generation commands, code samples, and descriptive text together throughout the article. Since it can be very app specific and (potentially) complex, it would be prudent to mention the benefits to some requirement planning before implementation.

Your system outline below your schema diagram is a good start to the reqs &/or schema sections. The schema section is also basically taken care of with your diagram. It would be easy enough to add the prisma models.

I would like to include some of @dthyresson's thoughts on thinking out the system requirements. We could possibly take some snippets from the Community Approaches/Mentions section above. He has already given some good tips on thinking through system requirements.

Regarding any generator/code inclusion: You could probably run down just one side of the system at certain points. Meaning, only show code for the player and instruct the user to repeat for the coach. At other sections, it would be assumed that the coach side had been generated based on those instructions.

I would like to see a varied section on authorization / data access patterns or options. Meaning - what are solid ways to ensure data is not leaked across tenants. I have seen services mentioned, directives maybe, direct query editing, prisma middleware, RLS... Is there a best way? We can always include the caveat that more complex options may be more suitable depending on system requirements.

If we were to write a How-To guide, should we make it about a more complicated system, like mine, or the most basic system we can think of?

I suggest sticking with yours, as you have outlined it above. It would be nice to provide a moderately complex, real world solution that could be pared down. A form of your system may be directly applicable to someone else's app. I might say it is only a step or so more complex than the blitz example. Also, having a more complex example may offer more room for future expansion of the How To.

Hah, a potential side benefit - you may get some nice ideas back in the form of questions/feedback from users that could be actionable in your live system.

Tobbe commented 2 years ago

Hah, a potential side benefit - you may get some nice ideas back in the form of questions/feedback from users that could be actionable in your live system.

This was my (now not so) secret master plan all along 🤣 😛

Thanks for your reply @pantheredeye. Let's see what @dthyresson's thoughts are 🙂

MichaelrMentele commented 1 year ago

I haven't read this yet, just dropping a quick drive by note -- may circle back but the simplest way to support multi-tenancy for us was to add a prisma middleware. We have a recursive middleware that injects the tenant (from the auth) so that tenancy is invisible to our application code by default. You can still pop off the hood easily and pass in tenant where desired.

chemalopezp commented 1 year ago

Adding to @MichaelrMentele comment, the most necessary feature to multi-tenancy is the Prisma middleware that would filter records depending on the organization(s) an user belongs (i.e. generators add that middleware to SDLs, etc.)

That said, in a second pass I would add to Redwood dbauth multi-tenancy features. I think many of us feel the current solutions to authenticate into organizations are running short, and that we might benefit from having that on-premise.

hakontro commented 1 year ago

I haven't read this yet, just dropping a quick drive by note -- may circle back but the simplest way to support multi-tenancy for us was to add a prisma middleware. We have a recursive middleware that injects the tenant (from the auth) so that tenancy is invisible to our application code by default. You can still pop off the hood easily and pass in tenant where desired.

@MichaelrMentele Any chance you can share, or expand on, that middleware ? Sounds useful and interesting.

dthyresson commented 1 year ago

Hi @hakontro you may be interested in the Prisma Client Extensions. I have an example that am working on to showcase Postgres Row-Level-Security RLS with Supabase here: https://github.com/dthyresson/prisma-extension-supabase-rls

The concept would be similar:

Of course, one would still need the Team and Organization models and user relationships, but hope this helps see where this "middleware" can help filter at the Prisma query level.

The repo has links to the relevant Prisma docs.

hakontro commented 1 year ago

Good tip, thanks a lot! I actually ran into the prisma client extensions somewhere else, and they seem to be a better fit than middleware for this.

Could you give me a pointer on how to safely "set team or org ids in context"? I'm implementing middleware to get a list of all orgs a user is part of as we speak, but I'm sure there's a better way!

dthyresson commented 1 year ago

@realStandal did a proof of concept here: https://github.com/realStandal/redwood-rls-demo/blob/main/api/src/plugins/prisma-auth.ts

That uses:

You could also, use Yoga's extendContext to add the team and org to the context can fetch and set in the client extension.

for an example of "extendContext" see: https://github.com/redwoodjs/redwood/blob/main/packages/graphql-server/src/plugins/useRedwoodAuthContext.ts

This is where the context.currentUser is set on global context as part of auth. You could "get team and org from headers or somewhere" and then in onContextBuilding use extendContext so that context.teamId etc would be set.

I'm working on a pattern for this as well -- and the constraint right now is RedwoodJS' tests and scenarios. For it to now which client to use when and where -- and if data-level security testing should be done in service tests or not.

Also, this PoC assumes a user belongs to a single team and org, and I think I'd want a user to belong to multiple teams,

Therefore, the context or those ids may not come from the currentUser but some other input, ie - what team is the user working in "right now", etc.

But, hope this can point you in the direction.

hakontro commented 1 year ago

Thanks, exactly what I needed. Seems to work now, I just

Prisma Extended Client fixes the rest

$allOperations({ args, query }) {
    args.where = 
    {
        orgId: Number(orgIdFromHeader),
        ...args.where,
      }
    return query(args)
}

but I'm running into problems with the generated services, because

export const addresses: QueryResolvers['addresses'] = () => {
  return db.address.findMany()
}

export const address: QueryResolvers['address'] = ({ id }) => {
  return db.address.findUnique({
    where: { id },
  })
}

findUnique doesn't support filtering by mulitple fields at once. findMany works as intended though. Worth thinking about when designing this!

MichaelrMentele commented 1 year ago

@hakontro hey we would be willing to share, @AlejandroFrias could give an overview perhaps in the forum and link here, I think I've shared source with the core team as well