search

appy-one / acebase-server

A fast, low memory, transactional, index & query enabled NoSQL database server for node.js that easily syncs with browser and node.js clients and servers
MIT License
32 stars 14 forks source link

readme

AceBase realtime database server

This repository is to setup an http endpoint for a local AceBase database instance. See AceBase for more information about AceBase databases and usage.

AceBase server webmanager

AceBase server Swagger UI

Getting started

Install the acebase-server npm package: npm install acebase-server (github, npm)

To test running an AceBase server with authentication on localhost:3000, execute the following commands

Creating an AceBase server

To launch an AceBase webserver using default settings

const { AceBaseServer } = require('acebase-server');
const dbname = 'mydb';
const server = new AceBaseServer(dbname, { host: 'localhost', port: 5757 });
server.on("ready", () => {
    console.log("SERVER ready");
});

NOTE: The above code will create an admin user with a generated password, which will be displayed once in the console output. Only randomly salted SHA512 hashes of user passwords are stored in the auth database, so it'll become fairly hard to sign into the admin account if you don't copy/paste the password somewhere. If you don't want to use a generated password, see Enable authentication below for info about how to supply a default password.

Enable SSL (https)

To run the server on a secure https connection, you can simply provide details about the location of your certificate files:

const settings = {
    host: 'mydb.example.com',
    port: 443,
    https: {
        certPath: './mycertificate.pem',
        keyPath: './myprivatekey.pem'
    }
};
const server = new AceBaseServer(dbname, settings);

You can either pass certPath and keyPath, or pfxPath and passphrase - depending on the type of certificate files you have.

Enable transaction logging

(NEW, BETA)

AceBase now supports transaction logging to facilitate sophisticated synchronization options and custom data recovery. Using cursors that indicate certain points in time, this allows for fast and easy synchronization of data between an AceBase server and multiple clients, or other server instances. This functionality is currently in BETA stage and will be tested extensively in the coming weeks.

To enable transaction logging on your database, add the transactions setting:

const server = new AceBaseServer(dbname, { 
    host: 'localhost', port: 443,
    transactions: { 
        log: true,      // Enable
        maxAge: 30,     // Keep logs of last 30 days
        noWait: false   // Data changes wait for log to be written
    }
};

Enable authentication

Authentication settings determine who is allowed access to your server. By enabling authentication, the server will allow users to sign in (and signup) and authorization rules to be defined for data being read/written to the database.

const settings = {
    host: 'mydb.example.com',
    port: 80,
    authentication: {
        enabled: true,
        allowUserSignup: false,
        defaultAccessRule: 'auth',
        defaultAdminPassword: '75sdDSFg37w5'
    }
}

The following authentication settings are available:

Setup authorization rules

If you enabled authentication, you can also define access rules for your data. Using rules, you can allow or deny specific (or anonymous) users read and/or write access to your data. These rules are similar to those used by Firebase, but they are not identical. AceBase has easy ".schema" rules allow for an easy and clean way to validate data being written, while ".validate" rules allow you to check existing data and use more advanced business logic. The rules can be saved to the rules.json file in your database directory, and can be overridden and extended programmatically from within your own code: ".validate" rules are coded in JavaScript (see Coding your rules). The default rules written to the file are determined by the defaultAccessRule authentication setting at the first server launch with authentication enabled.

The default rules.json file content is based on the value of the defaultAccessRule setting, possible values are:

When defaultAccessRule: "auth" is used, it will generate the following rules.json file:

{
    "rules": {
        ".read": "auth !== null",
        ".write": "auth !== null"
    }
}

When "allow" or "deny" is used, the ".read" and ".write" properties will be set to true or false respectively.

If you want to further restrict what data users can read and/or write to (RECOMMENDED!), you could edit the file as such, granting users read/write access to their own user node:

{
    "rules": {
        "users": {
            "$uid": {
                ".read": "auth.uid === $uid",
                ".write": "auth.uid === $uid"
            }
        }
    }
}

NOTE: Just like Firebase, access is denied by default if no rule is found for a target path. If an access rule is found at any location in the path, it will be used for any child path UNLESS your rule returns "cascade" (v1.17.0+). This means that, unlike Firebase, read and/or write access for child/descending paths CAN be overridden if you need to. "cascade" simply instructs the rule parser to postpone the decision making if the request is made on a child path, which is essentially as if there was no rule set for the current path. If the request is made on the rule path itself though, "cascade" will deny access because it is the last rule to execute and no access was granted.

For example, if you want to allow users read access to a path, and write access only for specific child path(s), use the following rules:

{
    "rules": {
        "shop_reviews": {
            "$shopId": {
                ".read": true,
                "$uid": {
                    ".write": "auth.uid === $uid"
                }
            }
        }
    }
}

Above rules enforces:

If you want to allow a specific user read/write access to a path, and only read access to a specific child path for all other users:

{
    "rules": {
        "users": {
            "$uid": {
                ".read": "auth?.uid === $uid ? 'allow' : 'cascade'",
                ".write": "auth.uid === $uid",
                "public": {
                    ".read": true
                }
            }
        }
    }
}

Above rules enforces:

NOTE that "allow" can be returned from a rule function instead of true, and "deny" instead of false, undefined or other falsy values.

Rule environment variables and functions

(NEW v1.17.0+)

Besides the auth variable used in above examples, there are other variables and functions you can use in your rule definitions. You can use these to determine if the data to be read or written conforms to your business logic. The following variables and functions are available in your .read and .write rules:

Note that instead of using the operation variable, you can also specify specific rules for each operation: the rules ".write": "true", ".set": "auth.uid !== null" allow all users to write data to nested paths, but restrict set operation to signed in users only. The same can be done with a single write rule: ".write": "operation !== 'set' || auth.uid !== null" or more verbose: ".write": "if (operation !== 'set') { return true; } else { return auth.uid !== null }"

Remember that above rules cascade to child paths, and are therefore enforced for all data being read from or written to child/descendant paths - once a rule allows or denies access, this cannot be overridden by rules defined on nested child paths. The only exception to this are .validate rules that are evaluated on the target paths being read from or written to. If a parent .write rule allows access, a .validate rule on the target path is still able to deny access.

Validate data being written

(NEW v1.17.0+)

In addition to the read/write rules explained above, AceBase now supports defining .validate rules. In contrast to .read and .write rules, .validate rules do not cascade and are enforced on the target path only. This allows you to validate the data being written to specific paths. .validate rules are executed after verifying that a .write rule grants access. They fully support JavaScript, so you can use the same checks you'd use in client side code to validate your data with. As a general rule, use .validate rules only if you have to perform checks that .schema rules can't handle, such as using current data in the database, or advanced type checks such as string length.

The following additional variables are available to '.validate' rules:

NOTE: keep in mind that the value for 'update' operations are partial objects to update the existing stored value with. If your .validate rule checks for property existence or their values, make sure you allow missing properties if the operation is 'update' (or use a .schema rule instead)

Example: validate writes to /widget

{
    "rules": {
        "widget": {
            ".write": true,
            // a valid widget must have attributes "color" and "size", but ignore update operations (partial objects!)
            ".validate": "operation === 'update' || ('color' in data && 'size' in data)",
            "size": {
                // the value of "size" must be a number between 0 and 99
                ".validate": "typeof data === 'number' && data >= 0 && data <= 99"
            },
            "color": {
                // the value of "color" must exist as a key in /valid_colors (eg /valid_colors/black)
                ".validate": "typeof data === 'string' && await exists(`/valid_colors/${data}`)"
            }
        }
    }
}

The above example could be combined with .schema rules to make these checks easier. See Schema validation below for more info.

{
    "rules": {
        "widget": {
            ".write": true,
            // add required "color" and "size" types to schema definition
            ".schema": "{ color: string; size: number }",
            "size": {
                // the value of "size" must be a number between 0 and 99
                ".validate": "data >= 0 && data <= 99"
            },
            "color": {
                // the value of "color" must exist as a key in /valid_colors (eg /valid_colors/black)
                ".validate": "await exists(`/valid_colors/${data}`)"
            }
        }
    }
}

Schema validation

AceBase server supports TypeScript-like schema definitions and validation. Once you've defined a schema for a path, all data being written must adhere to the set schema. Data to be stored/updated will be validated against the schema and denied or allowed accordingly.

There are 2 ways to can add schemas:

To ensure all users have a name (string), email (string) and language (either Dutch, English, German, French or Spanish), optionally a birthdate (Date) and address (custom object definition), add the following to your rules.json file:

{
    "rules": {
        "users": {
            "$uid": {
                ".read": "auth.uid === $uid",
                ".write": "auth.uid === $uid",
                ".schema": {
                    "name": "string",
                    "email": "string",
                    "language": "'nl'|'en'|'de'|'fr'|'es'",
                    "birthdate?": "Date",
                    "address?": {
                        "street": "string",
                        "city": "string",
                        "country": "string",
                        "geo?": {
                            "lat": "number",
                            "lon": "number"
                        }
                    }
                }
            }
        }
    }
}

You can also decide to split the schema up into multiple levels:

{
    "rules": {
        "users": {
            "$uid": {
                ".read": "auth.uid === $uid",
                ".write": "auth.uid === $uid",

                ".schema": {
                    "name": "string",
                    "email": "string",
                    "language": "'nl'|'en'|'de'|'fr'|'es'",
                    "birthdate?": "Date",
                    "address?": "Object"
                },

                "address": {
                    ".schema": {
                        "street": "string",
                        "city": "string",
                        "country": "string",
                        "geo?": "Object"
                    },

                    "geo": {
                        ".schema": {
                            "lat": "number",
                            "lon": "number"
                        }
                    }
                }
            }
        }
    }
}

And, if you prefer, schema definitions can be defined as strings instead:

{
    "address": {
        ".schema": "{ street: string, city: string, country: string, geo?: { lat: number, lon: number } }"
    }
}

Coding your rules

(NEW v1.17.0+)

Instead of writing your rules into the rules.json file on your server, it is also possible to configure them in your code. Any rule you set with code will override or augment existing rules found in the rules.json file, so can use both. Coding your rules has a number of advantages:

To add a rule in your code, use the following syntax:

server.setRule(path, ruleTypes, async (env) => { /* your rule code */ });

where:

The callback function is an async function that receives all available environment variables in the env argument: env.auth will contain the auth object, env.vars the vars object etc. If you use the ES6 destructuring syntax, you can use the same rule syntax from your rules.json file: async ({ auth }) => auth !== null

It is best to setup your rules while AceBase Server is starting up, before it is accepting connections. To do this, you can pass an init callback function to the server settings that is executed right before the http server is started:

const server = new AceBaseServer('mydb', {
    host: 'localhost',
    port: 4000,
    // ...
    async init(server) {
        // Allow read and write access to users' own data
        server.setRule('users/$uid', ['read', 'write'], ({ auth, vars }) => auth.uid === vars.$uid);

        // Limit user status to either 'online' or 'offline'
        server.setRule('users/$uid/status', 'validate', ({ data }) => ['online', 'offline'].includes(data));

        // Only allow following existing users
        server.setRule('users/$uid/following/$otherUid', 'validate', async ({ vars, exists }) => await exists(`/users/${vars.$otherUid}`));
    }
});

NOTE that you can use the env.vars object to get access to the values of named wildcards in your database paths. The values are also exposed as env.$name, but you can't use them this way in TypeScript because they can't be predefined in the env type.

NOTE now this init method is available, is is advisable to also move any server.extend, server.db.schema.set, server.configAuthProvider and server.router.get etc calls you'd have here!

Sending user e-mails [NEW]

If your app allows users to sign up, you'll need a way to verify their e-mail addresses, send welcome and password reset e-mails etc. AceBase Server does not send the e-mails itself (yet), but provides a way to handle this yourself:

const server = new AceBaseServer(('mydb', { host: 'localhost', port: 5757, authentication: { enabled: true, allowUserSignup: true }, email: { send: sendRequestedEmail } });

function sendRequestedEmail(request) {
    switch (request.type) {
        case 'user_signup': {
            console.log(`Should send an e-mail to ${request.user.email} to verify their e-mail address with code ${request.activationCode}`);
            break;
        }
        case 'user_reset_password': {
            console.log(`Should send an e-mail to ${request.user.email} to reset their password with code ${request.resetCode}`);
            break;
        }
    }
}

Any links you would add to the e-mails you send out should point to your own website/app, where you would handle account verification / password resets through an AceBaseClient connected to your server:

const resetCode = 'weydgaed7gjsdhfjadbsfadsfasq3w7dtuqwebd'; // eg from your_reset_url?code=weydga...
const newPassword = 'MyNewPassword'; // from user input on your page
try {
    await client.auth.resetPassword(resetCode, newPassword);
    // User can now sign in with their new password
}
catch(err) {
    // Something went wrong
}

OR, if you have direct access to your AceBaseServer instance from your website:

await server.resetPassword(req.ip, resetCode, newPassword);

To verify a user's email address:

const verificationCode = 'weydgaed7gjsdhfjadbsfadsfasq3w7dtuqwebd'; // eg from your_verify_url?code=weydga...
await client.auth.verifyEmailAddress(verificationCode);

OR, directly on your server:

await server.verifyEmailAddress(req.ip, verificationCode);

E-mail request examples

If you've configured the email settings with a function like sendRequestedEmail in the example above, your function will be called with e-mail requests with the following formats:

Request type user_signup:

{
    "type": "user_signup",
    "user": {
        "uid": "knq2aynt000dnc7bwt8gpj0c",
        "email": "me@appy.one",
        "displayName": "Ewout",
        "settings": {}
    },
    "ip": "127.0.0.1",
    "date": new Date("2021-04-20T13:27:46.937Z"),
    "activationCode": "eyJ2IjoxLCJjcyI6ImNlMGRlYTgxM...",
    "emailVerified": false,
    "provider": "acebase"
}

Request type user_reset_password:

{
    "type": "user_reset_password",
    "user": {
        "email": "me@appy.one",
        "uid": "knq3eq5w0005jg7b3vym9fgr",
        "settings": {},
        "displayName": "Ewout"
    },
    "ip": "127.0.0.1",
    "date": new Date("2021-04-21T07:19:26.588Z"),
    "resetCode": "eyJ2IjoxLCJjcyI6ImI4ZTE5MGE5NTA3YT..."
}

Request type user_reset_password_success:

{
    "type": "user_reset_password_success",
    "user": {
        "uid": "knq3eq5w0005jg7b3vym9fgr",
        "email": "me@appy.one",
        "displayName": "Ewout",
        "settings": {}
    },
    "ip": "127.0.0.1",
    "date": new Date("2021-04-21T10:15:28.148Z")
}

Using third party login providers

You can enable users to sign into your app through a third party login provider, such as Facebook, Google, Twitter etc. To enable this, follow these steps for each provider:

server.configAuthProvider('facebook', { client_id: '[your fb app_id]', client_secret: '[your fb app_secret]', scopes: [/* Any additional scopes, such as 'user_birthday' */] });

The result object will also contain the provider's access_token and refresh_token in case you want to make custom calls to the provider's API. If you want to keep the provider's access_token active, you will have to call client.auth.refreshAuthProviderToken before it expires:

const keepAlive = (provider) => {
    // Schedule token refresh 1 minute before it expires
    const refreshMs = (provider.expires_in - 60) * 1000;
    setTimeout(() => {
        client.auth.refreshAuthProviderToken(provider.refresh_token)
        .then(result => {
            keepAlive(result.provider); // Schedule again
        })
    }, refreshMs);
};

client.auth.finishAuthProviderSignIn(callbackResult)
.then(result => {
    // schedule a token refresh:
    keepAlive(result.provider);
});

Currently implemented auth providers are:

*NOTE: Facebook access tokens are short-lived by default, but will be exchanged for a long-lived (60 day) access token upon refresh. If you need to keep the Facebook access token active, execute refreshAuthProviderToken immediately after finishAuthProviderSignIn, and keep refreshing every time the user starts your app. Once a Facebook access token has expired, it cannot be refreshed and the user will have to sign in again.

Add cloud functions to handle data changes

You can add "cloud functions" to perform custom tasks upon data changes. You can do this in 2 ways:

// Monitor images being added by users, resize them to multiple sizes: server.db.ref('uploads/images').on('child_added', async snap => { const image = snap.val(); const resizedImages = await createImageSizes(image); // Some function that creates multiple image sizes in a worker thread const targetRef = await db.ref('images').push(resizedImages); // Store them somewhere else await snap.ref.remove(); // Remove original upload });


* Connect to your server with an AceBaseClient in a separate nodejs app, sign in as admin (or other user with rights to the data you want to read/write), add your custom event handlers.

```js
const db = new AceBaseClient({ host: 'localhost', port: 5757, dbname: 'mydb', https: false });
await db.ready();
await db.auth.signIn('admin', 'thepassword');

// Monitor images being added by users, resize them to multiple sizes:
db.ref('uploads/images').on('child_added', async snap => {
    const image = snap.val();
    const resizedImages = createImageSizes(image); // Some function that creates multiple image sizes
    const targetRef = await db.ref('images').push(resizedImages); // Store them somewhere else
    await snap.ref.remove(); // Remove original upload
});

Extending the server API

To extend the server's endpoints, there are 2 options:

Using Express router

Simply add your routes with server.router.get, .put, .post etc. You can use req.user to check if it is an authenticated request - your client code must add the Authorization: Bearer [token] header to requests in order for this to work.

server.router.get('/membersonly', (req, res) => {
    if (!req.user) {
        return res.status(403).send('Members only');
    }
    res.send(`Welcome ${req.user.display_name}`);
});

If you want to create your own router to bind custom routes to:

const myRouter = server.app.createRouter();
// Bind the new router to /myroutes
server.router.use('/myroutes', myRouter);
// Add routes to own router
myRouter.get('custom1', handleMethod1); // /myroutes/custom1
myRouter.get('custom2', handleMethod1); // /myroutes/custom2

Using server.extend

You can add your own custom API functions to the server with the server.extend(method, path, handler) method:

const server = new AceBaseServer(...);
await server.ready();

const _quotes = [...];
server.extend('get', 'quotes/random', (req, res) => {
    const index = Math.round(Math.random() * _quotes.length);
    res.send(quotes[index]);
});
server.extend('get', 'quotes/specific', (req, res) => {
    const index = req.query.index;
    res.send(quotes[index]);
});
server.extend('post', 'quotes/add', (req, res) => {
    const quote = {
        text: req.body.text,
        author: req.body.author
    };
    quotes.push(quote);
    res.send('thanks!');
});

You can then call your API methods on the client side as follows:

// Get random quote:
client.callExtension('get', 'quotes/random')
.then(quote => {
    console.log(`Got random quote: `, quote);
});

// Get specific quote:
client.callExtension('get', 'quotes/specific', { index: 15 })
.then(quote => {
    console.log(`Got quotes[15]: `, quote);
});

// Add a quote
client.callExtension('post', 'quotes/add', { 
    text: 'The greatest glory in living lies not in never falling, but in rising every time we fall', 
    author: 'Nelson Mandela' })
.then(result => {
    // "thanks!"
});

Running in a cluster

It is possible to run your AceBaseServer in a cluster so you can utilize more than just the 1 CPU a Node.js application typically uses. To make sure all instances can read and write to the same database files safely they will have to be able to communicate with each other. This is done using a technique called interprocess communication (IPC). Node.js' built-in cluster functionality provides IPC channels to communicate between master and worker processes, but if you want to run a pm2 cluster or want multiple instances running in the cloud, you'll have to use an external IPC server. See AceBase IPC Server for more information about the different clustering solutions and code examples.

Connecting to a server

See acebase-client on npm or github

Upgrade notices

v0.9.7: user accounts and server logs are now stored in the target database, instead of in a seperate auth database. If you upgraded from version 0.9.6 or lower and want to keep using the existing auth database, start the server with seperateDb: true in the options.