bajtos
commented
4 years ago I don't have bandwidth to look into this topic in full, but would like to dump few ideas & pointers to make it easier for others to do the research.
Conceptually, querying a database table involves the following actors:
- Model describes the shape of the data (table columns) and metadata (table name, DDL schema name, etc.)
- Repository binds a model with a datasource, registers the model with the datasource. At the moment, this involves converting LB4 model definition to LB3/juggler style.
- Datasource represents a database client and maintains a pool of open database connections. The datasource is configured with a connector to use (PostgreSQL, MongoDB, etc.), connection settings (host/port, credentials) and most importantly the database name to use. Most (if not all) LoopBack connectors require each datasource to use only one database name, it's not possible to switch between databases at runtime.
- The database server.
In a typical LB4 application, each model is associated with the same Repository class and the same DataSource instance - the wiring is static.
To enable multi-tenancy, we want to make this wiring dynamic. Depending on the current user, we want to use a different repository and/or datasource configuration.
Lightweight tenant isolation using schemas
In this setup, the authentication layer and all tenants share the same database name and use the same credentials (database user) to access the data. We have 1+N schemas defined in the database: the first schema is used by the authentication layer, plus we have one schema for each tenant. All database queries will use the same LB datasource and thus share the same connection pool.
Implementation wise, we need to tweak the way how a LB4 model is registered with a datasource. Instead of creating the same backing juggler model for all users, we want to create tenant-specific juggler models.
Conceptually, this can be accomplished by tweaking the Repository constructor.
export class ProductRepository extends DefaultCrudRepository<
Product,
typeof Product.prototype.id
> {
constructor(
@inject('datasources.db') dataSource: juggler.DataSource,
@inject(SecurityBindings.USER) currentUser: UserProfile,
) {
super(
// model constructor
Product,
// datasource to use
dataSource,
// new feature to be implemented in @loopback/repository:
// allow repository users to overwrite model settings
{schema: currentUser.name},
);
}
}Datasource-based tenant isolation
If schema-based isolation is not good enough (or not supported by the target database), or if we don't want tenants to share the same database connection pool, then we can wire our application to use a different datasource for each tenant. This approach unlocks new option for tenant isolation, for example it's possible to use different database & credentials for each tenant.
LB4 applications are already using Dependency Injection to obtain the datasource instance to be provided to Repository constructors. By default, a datasource is bound in a static way and configured to a singleton, see app.dataSource().
To support multi-tenancy, we need to rework the resolution of datasources to be dynamic, based on the current user.
Let's start from outside. To make it easy to inject the tenant-specific datasource, let's keep the same datasource name (binding key), e.g. datasources.tenantData, but implement dynamic resolution of the datasource value. The idea is to rework the datasource class scaffolded by lb4 datasource into a Provider class.
import {inject} from '@loopback/core';
import {juggler} from '@loopback/repository';
const config = {
name: 'tenantData',
connector: 'postgresql',
// ...
};
export class TenantDataSourceProvider implements Provider<TenantDataSource > {
constructor(
@inject('datasources.config.tenant', {optional: true})
private dsConfig: object = config,
@inject(SecurityBindings.USER)
private currentUser: UserProfile,
) {}
value() {
const config = {
...this.dsConfig,
// apply tenant-specific settings
schema: this.currentUser.name
};
// Because we are using the same binding key for multiple datasource instances,
// we need to implement our own caching behavior to support SINGLETON scope
// I am leaving this aspect as something to figure out as part of the research
const cached = // look up existing DS instance
if (cached) return cached;
const ds = new TenantDataSource(config);
// store the instance in the cache
return ds;
}
}
export class TenantDataSource extends juggler.DataSource {
static dataSourceName = 'tenant';
// constructor is not needed, we can use the inherited one.
// start/stop methods are needed, I am skipping them for brevity
}There are different ways how to implement caching of per-tenant datasources. Ideally, I would like to reuse Context for that. It turns out this is pretty simple!
We want each tenant datasource to have its own datasource name and binding key. To allow repositories to obtain the datasource via @inject, we can implement a "proxy" datasource provider that will be resolved using one of the name datasources.
export class TenantDataSourceProvider implements Provider<TenantDataSource> {
private dataSourceName: string;
private bindingKey: string;
constructor(
@inject('datasources.config.tenant', {optional: true})
private dsConfig: object = config,
@inject(SecurityBindings.USER)
private currentUser: UserProfile,
@inject.context()
private currentContext: Context,
@inject(CoreBindings.APPLICATION_INSTANCE)
private app: Application,
) {
this.dataSourceName = `tenant-${this.currentUser.name}`;
this.bindingKey = `datasources.${this.dataSourceName}`;
}
value() {
if (!this.currentContext.isBound(this.bindingKey)) {
this.setupDataSource();
}
return this.currentContext.get<juggler.DataSource>(this.bindingKey);
}
private setupDataSource() {
const resolvedConfig = {
...this.dsConfig,
// apply tenant-specific settings
schema: this.currentUser.name,
};
const ds = new TenantDataSource(resolvedConfig);
// Important! We need to bind the datasource to the root (application-level)
// context to reuse the same datasource instance for all requests.
this.app.bind(this.bindingKey).to(ds).tag({
name: this.dataSourceName,
type: 'datasource',
namespace: 'datasources',
});
}
}
export class TenantDataSource extends juggler.DataSource {
// no static members like `dataSourceName`
// constructor is not needed, we can use the inherited one.
// start/stop methods are needed, I am skipping them for brevity
}The code example above creates per-tenant datasource automatically when the first request is made by each tenant. This should provide faster app startup and possibly less pressure on the database in the situation when most tenants connect use the app only infrequently. On the other hand, any problems with a tenant-specific database connection will be discovered only after the first request was made, which may be too late. If you prefer to establish (and check) all tenant database connection right at startup, you can move the code from setupDataSource to a boot script and invoke it for each known tenant.
Open questions:
- Where to put this new provider file and how to register the provider at boot time? We don't want
@loopback/bootto treat it as a regular datasource file, because we want to bind the provider, not the datasource class. - How to enable
lb4 repository(and other) commands to recognize our special datasource and include it in the list of known datasources?
A possible solution is to enhance @loopback/boot to support datasource providers (e.g. using the convention datasources/{name}.datasource.provider.ts) and also enhance @loopback/cli to recognize these providers too.
emonddr
raymondfeng
King-Success
fredhansen25
Narrator
mitsos1os
pookdeveloper
any solution ? Thanks
Any example please?
mbnoimi
nehaked
asim-jana
Recently, several people asked about implementing dynamic schema selection to enable schema-base multi-tenancy, where tenant isolation is achieved via DDL Schemas. (If you are not familiar with DDL schemas then you can learn the basics e.g. in PostgreSQL docs).
I am opening this Epic to do discuss possible solutions, implement necessary improvements and document how to implement multi-tenancy, possibly including an example app.
Related discussions:
Aspects to consider:
lb4 datasource@loopback/boot(if needed) or move the registration to a different place (e.g. from a datasource file to a boot script)