Seeding/updating design docs into DBs post creation

[SukantGujar]

So, during app life-cycle, design docs may be added/edited against private and shared DBs. Is there a mechanism in SL right now which can automatically update them into the existing DBs during app startup?

Why this is important? The DBs are created and maintained by SL. It needs the design docs in a certain format (seed) and already has support to add them in the created DBs. However as the app develops further, there will be changes to these docs and it'd be tedious to have an update mechanism outside of SL.

As of now, my test app is in early stages of development, so it is trivial for me to reset the SL DBs and begin again, in case I need new design docs seeded, but this is not scale-able and I'd soon need to maintain new/changed design docs in both the CouchDB format as well as seed format and would need to come up with a process of updating these changes.

[colinskow]

When you create a user or add a new database to an existing user, SuperLogin automatically seeds the design docs that you specify in your config. If you update the design docs later you will need to manually reseed them.

Maintaining DBs seeded with the correct design docs as they update is complex and outside the scope of SuperLogin. I would love to see an external plugin to help with the tedious tasks of managing userDBs.

SL uses pouchdb-seed-design behind the scenes.

[SukantGujar]

So, while trying to solve this for my app, I went through the code of addUserDB and pouch-seed-design. Apparently, this flow is already taking care of -

1. Ignoring existing DBs.
2. Seeding only changed docs in the said DB. For me, simply invoking superlogin.addUserDB in the login event should work, as it will not recreate the DB but still update only changed/new design docs. However, I think with this minimal logic, Superlogin can internally take care of this need without much complication. Please provide your thoughts. I can request a PR.

This feature is a certain requirement for my application, one way or the other. Its a given thing that as new features are added, user DBs will have new designs and modifications. I'd hope you could allocate some of your time to reevaluate this for Superlogin.

[colinskow]

I've been giving some thought to this and we definitely need a user database manager. This is not only to keep the design docs up to date when they change, but also to add and remove user roles and change permissions.

[peteruithoven]

We also have the need for this, if only to more easily develop a validate_doc_update function. So I've started on a script we can run as util. A pitty is that most of it's logic is almost a direct copy of user.js's addUserDBs and dbauth/index.js's addUserDB functions. It currently only creates & updates shared databases and it doesn't update existing sessions. (See my question here: https://github.com/colinskow/superlogin/issues/152#issuecomment-299838523)

const SuperLogin = require('superlogin');
const superloginConfig = require('../superlogin.config.js');
const DBAuth = require('superlogin/lib/dbauth');
const PouchDB = require('pouchdb');
const seed = require('pouchdb-seed-design');
const util = require('superlogin/lib/util');

const superlogin = new SuperLogin(superloginConfig);
const { config } = superlogin;
const dbAuth = new DBAuth(config/* , userDB, couchAuthDB*/);

if (!config.getItem('userDBs.defaultDBs')) {
  console.log('no userDBs.defaultDBs defined');
  process.exit();
}

const cloudant = config.getItem('dbServer.cloudant');
let adapter;
if (cloudant) {
  adapter = require('superlogin/lib/dbauth/cloudant');
} else {
  throw new Error('CouchDB not supported');
}

let allDBs = [];
function addDBs(dbs, type) {
  if (!Array.isArray(dbs)) return;
  dbs.forEach(dbName => {
    const dbConfig = dbAuth.getDBConfig(dbName);
    dbConfig.type = type; // override type
    if (type !== 'shared') return; // private not supported yet
    allDBs.push(dbConfig);
  });
}
addDBs(config.getItem('userDBs.defaultDBs.private'), 'private');
addDBs(config.getItem('userDBs.defaultDBs.shared'), 'shared');

function updateDBs(dbs) {
  console.log('updateDBs: ', dbs);
  return asyncIterator(dbs, dbConfig => {
    console.log('dbConfig: ', dbConfig);
    const { name, adminRoles, memberRoles, designDocs } = dbConfig;
    console.log('updating: ', name);
    let newDB;
    return dbAuth.createDB(name)
      .then(response => {
        if (response !== false) console.log('Created db: ', name);
        newDB = new PouchDB(util.getDBURL(config.getItem('dbServer')) + '/' + name);
        return adapter.initSecurity(newDB, adminRoles, memberRoles);
      })
      .then(() => {
        // Seed the design docs
        if (designDocs && designDocs instanceof Array) {
          // designDocs.forEach(ddName => {
          console.log('designDocs: ', designDocs);
          return asyncIterator(designDocs, ddName => {
            console.log('ddName: ', ddName);
            const dDoc = dbAuth.getDesignDoc(ddName);
            console.log('dDoc: ', dDoc);
            if (dDoc) {
              return seed(newDB, dDoc);
            } else {
              console.warn('Failed to locate design doc: ' + ddName);
            }
          });
        }
      });
  });
}

updateDBs(allDBs).then(() => {
  process.exit();
});

function asyncIterator(args, callback) {
  return new Promise((resolve, reject) => {
    const results = [];
    if (args.length === 0) resolve(results);
    const clone = [...args];
    function loop(arg) {
      callback(arg).then(function (result) {
        results.push(result);
        if (clone.length > 0) {
          loop(clone.shift());
        } else {
          resolve(results);
        }
      });
    }
    loop(clone.shift());
  });
}

[peteruithoven]

I think I've found a way to update current session API keys in databases and re-use the existing dbauth/index.js's addUserDB function. The following function should update all databases according to configuration. Because everything happens in series (not parallel) this should also work for bigger projects. https://gist.github.com/peteruithoven/b8fd1f60ae526c849479e50a38709590#file-updatedbs-js

Curious what you guys think.

[peteruithoven]

Would it be interesting to add this to the general API, just like removeExpiredKeys?