FilipNest
commented
8 years ago Database abstraction layer
This is now done and seems to be working. Nothing in core should break if this is merged in and you shouldn't have to make any site changes (possible breakages in custom modules are described below with fixes).
There are certainly future improvements (https://www.npmjs.com/package/waterline was mentioned that could open up Iris to all major databases without us having to re-write the implementations) but the system can be expanded in the future now the API structure is in.
Even if we carry on using MongoDB for most projects, splitting the database-specific code out makes future development easier.
Possible breaking changes:
iris.dbCollectionsis nowiris.entityTypesand only contains an object of entity types and their schema. It's no longer a list of mongoose collection objects. There aren't any calls to it in the core code anymore so this will only matter if you have used it directly in a custom module./api/entitySchemaand/api/entitySchema/:typeare now in a slightly different format. I'm not sure if they were used for testing or any custom integrations so this may be fine (the Drupal integration will need updating if it's still in use and uses this API). I think the new format makes more sense (and allows for our new object and array of object types). All the data is still there, just named differently.- Any custom/contrib modules that call MongoDB or Mongoose functions directly need to either be updated to use hook_entity_fetch, hook_entity_update and hook_entity_create or include Mongoose and do it all manually. If they go down the latter approach, you should check
iris.config.dbEngineto see if MongoDB is enabled.
How it works:
- A new site configuration parameter,
dbEnginesays which database hooks should be used (databases can now use hooks as the database stuff has been moved to the end of the boot process). - If this isn't set, it currently defaults to "mongodb" so it doesn't break any sites.
- A database module should defines the following hooks (the parameter at the end is the dbEngine):
hook_db_connect__(connects to the database)hook_db_schema__(registers schema/collections/tables)hook_db_createEntity__hook_db_updateEntity__hook_db_deleteEntity__hook_db_deleteSchema__(deletes schema/collections/tables)
- These hooks should not be invoked directly in most cases. They're called by the hooks and systems we've used until now where validation and other functions still live.
- The system has two database engines in core. MongoDB and nedb. Which of these is active is set using the dbEngine config parameter. Both are currently required in core. They don't have to be but we need to work out some way of running the system without a database active before they (and other database implementations) can be moved out. You can't enable database modules through the UI if the UI requires a database.
nedb
nedb is a JavaScript only lightweight database that does most of its work in memory. It uses the same syntax as MongoDB. It handles persistence by storing the data in JSON files.
Main advantages (and why I think it should be in core as an option):
- Doesn't need a database setting up with server connection details etc. So
npm install irisjswill do it all. This is superb for getting people to try Iris out. - Stores data in files that can be easily deleted, edited or even put in Git.
Have a go by putting the "dbEngine":"nedb" in a site's config.
One really good use case for it is testing as not requiring a MongoDB setup may make a lot of tests easier to run. Tests could just delete the database files when finished.
facascante
Part of issue #274 (MongoDB decoupling)
See comment below for the latest/best explanation.
Doing this early on as it needs lots of testing and will make the second stage of the database decoupling task a lot easier.I've gone through and replaced calls to Mongoose collections (find,update,count,get entity types) with the relevant hooks.I've also renamediris.dbSchemaConfigtoiris.entityTypeswhich should now be the main way of getting a list of entity types and their schema. Instead ofiris.dbSchemaConfigoriris.dbCollections. I've replaced all uses of these.I did tests on functionality I replaced but if someone could re-test it'd be very useful. Particularly the one time login link stuff as that might be broken (I've haven't used that functionality yet).There is one TODO regarding the API calls to get entityType schema as currently these schema don't show the fieldType information (just the name of the field type). Probably need to pull that in. The only places that use this are the two REST API queries in entity.js I think we used for the Drupal integration. If you're using these two API calls, let me know how and I'll try to get the iris.entityTypes structure to match that part as well. Worth looking into anyway now that we've got object fields, field collections and arrays of objects.This is now fixed.