Closed thibaultcha closed 9 years ago
Doing such a schema change could also fix this ugly problem which is plugins selection "fixed" by bb838907b436964f9694cf4392fec98e01b7fab9
I am not happy by the way we handle credentials and I think we should think more carefully how we want to support this. I think having something like the following schema could make sense:
We do have the following generic core entities:
accounts
, the base entity who owns of zero or more applications
. applications
, generic credential holder, that can hold different credential typesEach authentication plugin can create a credential type on the datastore, like:
query
basic
header
ldap
oauth
This means that we need to introduce the possibility for plugins to edit the datastore during their installation using a DSL.
If plugins can modify the datastore, that should be done with a DSL instead of plain simple SQL, to avoid doing illegal operations on the datastore. An example could be:
return { create = {
{
type = "table",
name = "ldap",
properties = [[
id uuid,
key text,
created_at timestamp,
PRIMARY KEY (id)
]]
},
{
type = "datatype",
name = "ldap_credential",
properties = [[
public text,
secret text
]]
}
}
}
By having the DSL we're limiting the number of operations that the plugin can execute on the datastore, like deleting other tables, or modifying existing data.
We could also implement a rollback function to execute DELETE
statements on whatever datastore entity has been created during the provisioning of the plugin (it needs to be explicit because it could cause loss of data).
This is even nicer, the action to create
those entities could be implicit, and we don't accept DB-specific constructs to support in the future any other datastore.
{
entities = {
{
name = "ldap",
properties = {
{ name = "id", type = "id" },
{ name = "key", type = "string" },
{ name = "created_at", type = "timestamp" },
{ name = "type", type = "ldap_credential"}
{ primary = "id"}
}
},
{
name = "ldap_credential",
properties = {
{ name = "public", type = "string", unique = true },
{ name = "secret", type = "string" }
}
}
}
}
The example above is a quick demonstration, but a DSL like this could technically be ported to any datastore without having to update the plugins if a new DAO is being introduced. The DAO will take care of translating the DSL to an executable statement.
And as long as the DSL is verbose enough, the DAO can then decide to handle edge-cases (like treating child entities like ldap_credential
as datatypes
in Cassandra, or just another table
in other datastores - it's up to the DAO).
On a side note the more I look at the DSL above, the more it resembles the schemas
we already have. If we decide to implement a DSL to DB translation, I wonder if we can automatically generate the migration script file by parsing all of the schemas, thus automating the creation of migrations files.
set
, ie: applications of an accounts.I like the idea, just time consuming for DAOs to implement. Otherwise:
plugins
table.drop
method @thefosk. It's up to us to distribute valid, trusted plugins.Regarding the DSL, a few points:
basicauth.keys
. This is a good idea for two reasons:
drop
or other reserved calls just to some special packages. Thus we can block every call that comes from the kong.plugins
package.Too bad we're not using PostgreSQL: http://leafo.net/lapis/reference/database.html#database-schemas
A great contribution would be implementing a Cassandra adapter to Lapis as mentioned in #80.
Cassandra 3.0 will support this: https://issues.apache.org/jira/browse/CASSANDRA-8473
Following the discussion we had yesterday, here are the decisions we took:
accounts:
renamed to consumers
.
id
: sameprovider_id
: renamed to custom_id
(same purpose) required if no username
username
: required if no provider_id
extra
: Maybe a field for extra informationsapis
: nothing newplugins
:
consumers
tableconfiguration entry
of that plugin, linked to an api
and optionally, a consumer
. This allows a plugin to be enabled on an API, as well as being overridable for a specific consumer
.We started talking about having a whitelist/blacklist for configuration entries (to be able to enable/disable a configuration entry for a lot of apis
/consumers
at once, but this ran into implementation issues as illustrated in the following picture.
This discussion was related to #50 (Plugins system), #91 (refactor applications), #93 (Plugins API), #98 (Better API routing)
Here is a pic of the whiteboard:
Improvements described in the previous comment are implemented, appart from:
Those things need to be done in order to provide a good development environment for plugins but will be part of another discussion: #93.
Plugins do expand the API routes, it has been implemented: https://github.com/Mashape/kong/blob/master/kong/api/app.lua#L76
We are waiting for the DAO part to have complete separation.
Mainly just writing thoughts down here, and trying to discuss the limitations of a future schema. Not a priority at the moment.
The current schema was built using different column families for accounts, applications, apis, plugins. We should probably handle relations the way Cassandra handles them:
Relations
Here, the index would allow us to query the accounts table by application's values (especially
public_key
, as it is the only value that will be queried), but it needs to happen like this:public_key
, like currentlyThe same applies for plugins. They are currently a table on their own, but a plugin is attached to an API, and optionally to an application.
Community plugins
Plugins from the community will have to use the
value
property and encode their data to store things. We could provide them with a way of creating a table, or a UDT.