search

coresmart / persistencejs

persistence.js is an asynchronous Javascript database mapper library. You can use it in the browser, as well on the server (and you can share data models between them).
http://persistencejs.org
1.73k stars 240 forks source link

readme

persistence.js

persistence.js is a asynchronous Javascript object-relational mapper library. It can be used both in the web browser and on the server using node.js. It currently supports 4 types of data stores:

There is also an experimental support for Qt 4.7 Declarative UI framework (QML) which is an extension to JavaScript.

For browser use, persistence.js has no dependencies on any other frameworks, other than the Google Gears initialization script, in case you want to enable Gears support.

Plug-ins

There are a few persistence.js plug-ins available that add functionality:

A Brief Intro to Async Programming

In browsers, Javascript and the web page's rendering engine share a single thread. The result of this is that only one thing can happen at a time. If a database query would be performed synchronously, like in many other programming environments like Java and PHP the browser would freeze from the moment the query was issued until the results came back. Therefore, many APIs in Javascript are defined as asynchronous APIs, which mean that they do not block when an "expensive" computation is performed, but instead provide the call with a function that will be invoked once the result is known. In the meantime, the browser can perform other duties.

For instance, a synchronous database call call would look as follows:

var results = db.query("SELECT * FROM Table");
for(...) { ... }

The execution of the first statement could take half a second, during which the browser doesn't do anything else. By contrast, the asynchronous version looks as follows:

db.query("SELECT * FROM Table", function(results) {
  for(...) { ... }
});

Note that there will be a delay between the db.query call and the result being available and that while the database is processing the query, the execution of the Javascript continues. To make this clear, consider the following program:

db.query("SELECT * FROM Table", function(results) {
  console.log("hello");
});
console.log("world");

Although one could assume this would print "hello", followed by "world", the result will likely be that "world" is printed before "hello", because "hello" is only printed when the results from the query are available. This is a tricky thing about asynchronous programming that a Javascript developer will have to get used to.

Using persistence.js in the browser

Browser support

(The following is being worked on:) Internet Explorer is likely not supported (untested) because it lacks __defineGetter__ and __defineSetter__ support, which persistence.js uses heavily. This may change in IE 9.

Setting up

bower install persistence

Add a <script> to your index.html:

lib/persistence.js needs to be added, as well as any data stores you want to use. Note that the mysql and websql stores both depend on the sql store. A typical setup requires you to add at least lib/persistence.js, lib/persistence.store.sql.js and lib/persistence.store.websql.js as follows:

<script src="https://github.com/coresmart/persistencejs/raw/master/bower_components/persistencejs/lib/persistence.js"></script>
<script src="https://github.com/coresmart/persistencejs/raw/master/bower_components/persistencejs/lib/persistence.store.sql.js"></script>
<script src="https://github.com/coresmart/persistencejs/raw/master/bower_components/persistencejs/lib/persistence.store.websql.js"></script>

If you want to use the in-memory store (in combination with localStorage) you also need the persistence.store.memory.js included.

Copy directories you will need following almost the same instructions above.

Setup your database

You need to explicitly configure the data store you want to use, configuration of the data store is store-specific. The WebSQL store (which includes Google Gears support) is configured as follows:

persistence.store.websql.config(persistence, 'yourdbname', 'A database description', 5 * 1024 * 1024);

The first argument is always supposed to be persistence. The second in your database name (it will create it if it does not already exist, the third is a description for you database, the last argument is the maximum size of your database in bytes (5MB in this example).

Setting up for Cordova with SQLitePlugin/WebSQL

Use following if you want to use persistencejs in a Cordova mobile app and you plan to use the Cordova SQLitePlugin:

persistence.store.cordovasql.config(
  persistence,
  'yourdbname',
  '0.0.1',                // DB version
  'My database',          // DB display name
  5 * 1024 * 1024,        // DB size (WebSQL fallback only)
  0,                      // SQLitePlugin Background processing disabled
  2                       // DB location (iOS only), 0 (default): Documents, 1: Library, 2: Library/LocalDatabase
                          //   0: iTunes + iCloud, 1: NO iTunes + iCloud, 2: NO iTunes + NO iCloud
                          //   More information at https://github.com/litehelpers/Cordova-sqlite-storage#opening-a-database
);

For more information on the SQLitePlugin background processing please refer to the SQLitePlugin readme.

The Cordova support in persistencejs will try to work with the SQLitePlugin if it is loaded; if not it will automatically fall back to WebSQL.

Please note that to use Cordova store, you must use the master branch, because it is not included up to release v0.3.0.

The in-memory store

The in-memory store is offered as a fallback for browsers that do not support any of the other supported stores (e.g. WebSQL or Gears). In principal, it only keeps data in memory, which means that navigating away from the page (including a reload or tab close) will result in the loss of all data.

A way around this is using the persistence.saveToLocalStorage and persistence.loadFromLocalStorage functions that can save the entire database to the localStorage, which is persisted indefinitely (similar to WebSQL).

If you're going to use the in-memory store, you can configure it as follows:

persistence.store.memory.config(persistence);

Then, if desired, current data can be loaded from the localStorage using:

persistence.loadFromLocalStorage(function() {
  alert("All data loaded!");
});

And saved using:

persistence.saveToLocalStorage(function() {
  alert("All data saved!");
});

Drawbacks of the in-memory store:

Schema definition

A data model is declared using persistence.define. The following two definitions define a Task and Category entity with a few simple properties. The property types are based on SQLite types, specifically supported types are (but any SQLite type is supported):

Example use:

var Task = persistence.define('Task', {
  name: "TEXT",
  description: "TEXT",
  done: "BOOL"
});

var Category = persistence.define('Category', {
  name: "TEXT",
  metaData: "JSON"
});

var Tag = persistence.define('Tag', {
  name: "TEXT"
});

The returned values are constructor functions and can be used to create new instances of these entities later.

It is possible to create indexes on one or more columns using EntityName.index, for instance:

Task.index('done');
Task.index(['done', 'name']);

These indexes can also be used to impose unique constraints :

Task.index(['done', 'name'],{unique:true});

Relationships between entities are defined using the constructor function's hasMany call:

// This defines a one-to-many relationship:
Category.hasMany('tasks', Task, 'category');
// These two definitions define a many-to-many relationship
Task.hasMany('tags', Tag, 'tasks');
Tag.hasMany('tasks', Task, 'tags');

The first statement defines a tasks relationship on category objects containing a QueryCollection (see the section on query collections later) of Tasks, it also defines an inverse relationship on Task objects with the name category. The last two statements define a many-to-many relationships between Task and Tag. Task gets a tags property (a QueryCollection) containing all its tags and vice versa, Tag gets a tasks property containing all of its tasks.

The defined entity definitions are synchronized (activated) with the database using a persistence.schemaSync call, which takes a callback function (with a newly created transaction as an argument), that is called when the schema synchronization has completed, the callback is optional.

persistence.schemaSync();
// or
persistence.schemaSync(function(tx) { 
  // tx is the transaction object of the transaction that was
  // automatically started
});

There is also a migrations plugin you can check out, documentation can be found in docs/migrations.md file.

Mix-ins

You can also define mix-ins and apply them to entities of the model.

A mix-in definition is similar to an entity definition, except using defineMixin rather than just define. For example:

var Annotatable = persistence.defineMixin('Annotatable', {
  lastAnnotated: "DATE"
});

You can define relationships between mix-in and entities. For example:

// A normal entity
var Note = persistence.define('Note', {
  text: "TEXT"
});

// relationship between a mix-in and a normal entity
Annotatable.hasMany('notes', Note, 'annotated');

Once you have defined a mix-in, you can apply it to any entity of your model, with the Entity.is(mixin) method. For example:

Project.is(Annotatable);
Task.is(Annotatable);

Now, your Project and Task entities have an additional lastAnnotated property. They also have a one to many relationship called notes to the Note entity. And you can also traverse the reverse relationship from a Note to its annotated object.

Note that annotated is a polymorphic relationship as it may yield either a Project or a Task (or any other entity which is `Annotatable').

Note: Prefetch is not allowed (yet) on a relationship that targets a mixin. In the example above you cannot prefetch the annotated relationship when querying the Note entity.

Notes: this feature is very experimental at this stage. It needs more testing. Support for "is a" relationships (classical inheritance) is also in the works.

Creating and manipulating objects

New objects can be instantiated with the constructor functions. Optionally, an object with initial property values can be passed as well, or the properties may be set later:

var task = new Task();
var category = new Category({name: "My category"});
category.metaData = {rating: 5};
var tag = new Tag();
tag.name = "work";

Many-to-one relationships are accessed using their specified name, e.g.: task.category = category;

One-to-many and many-to-many relationships are access and manipulated through the QueryCollection API that will be discussed later:

task.tags.add(tag);
tasks.tags.remove(tag);
tasks.tags.list(tx, function(allTags) { console.log(allTags); });

Persisting/removing objects

Similar to hibernate, persistence.js uses a tracking mechanism to determine which objects' changes have to be persisted to the database. All objects retrieved from the database are automatically tracked for changes. New entities can be tracked to be persisted using the persistence.add function:

var c = new Category({name: "Main category"});
persistence.add(c);
for ( var i = 0; i < 5; i++) {
  var t = new Task();
  t.name = 'Task ' + i;
  t.done = i % 2 == 0;
  t.category = c;
  persistence.add(t);
}

Objects can also be removed from the database:

persistence.remove(c);

All changes made to tracked objects can be flushed to the database by using persistence.flush, which takes a transaction object and callback function as arguments. A new transaction can be started using persistence.transaction:

persistence.transaction(function(tx) {
  persistence.flush(tx, function() {
    alert('Done flushing!');
  });
});

For convenience, it is also possible to not specify a transaction or callback, in that case a new transaction will be started automatically. For instance:

persistence.flush();
// or, with callback
persistence.flush(function() {
  alert('Done flushing');
});

Note that when no callback is defined, the flushing still happens asynchronously.

Important: Changes and new objects will not be persisted until you explicitly call persistence.flush(). The exception to this rule is using the list(...) method on a database QueryCollection, which also flushes first, although this behavior may change in the future.

Dumping and restoring data

The library supports two kinds of dumping and restoring data.

persistence.dump can be used to create an object containing a full dump of a database. Naturally, it is adviced to only do this with smaller databases. Example:

persistence.dump(tx, [Task, Category], function(dump) {
  console.log(dump);
});

The tx is left out, a new transaction will be started for the operation. If the second argument is left out, dump defaults to dumping all defined entities.

The dump format is:

{"entity-name": [list of instances],
 ...}

persistence.load is used to restore the dump produced by persistence.dump. Usage:

persistence.load(tx, dumpObj, function() {
  alert('Dump restored!');
});

The tx argument can be left out to automatically start a new transaction. Note that persistence.load does not empty the database first, it simply attempts to add all objects to the database. If objects with, e.g. the same ID already exist, this will fail.

Similarly, persistence.loadFromJson and persistence.dumpToJson respectively load and dump all the database's data as JSON strings.

Entity constructor functions

The constructor function returned by a persistence.define call cannot only be used to instantiate new objects, it also has some useful methods of its own:

And of course the methods to define relationships to other entities:

Entity objects

Entity instances also have a few predefined properties and methods you should be aware of:

Query collections

A core concept of persistence.js is the QueryCollection. A QueryCollection represents a (sometimes) virtual collection that can be filtered, ordered or paginated. QueryCollections are somewhate inspired by Google AppEngine's Query class. A QueryCollection has the following methods:

Query collections are returned by:

Example:

var allTasks = Task.all().filter("done", '=', true).prefetch("category").order("name", false).limit(10);

allTasks.list(null, function (results) {
    results.forEach(function (r) {
        console.log(r.name)
        window.task = r;
    });
});

Using persistence.js on the server

Installing persistence.js on node is easy using npm:

npm install persistencejs

Sadly the node.js server environment requires slight changes to persistence.js to make it work with multiple database connections:

An example node.js application is included in test/node-blog.js.

Setup

You need to require two modules, the persistence.js library itself and the MySQL backend module.

var persistence = require('persistencejs');
var persistenceStore = persistence.StoreConfig.init(persistence, { adaptor: 'mysql' });

Then, you configure the database settings to use:

persistenceStore.config(persistence, 'localhost', 3306, 'dbname', 'username', 'password');

Subsequently, for every connection you handle (assuming you're building a sever), you call the persistenceStore.getSession() method:

var session = persistenceStore.getSession();

This session is what you pass around, typically together with a transaction object. Note that currently you can only have one transaction open per session and transactions cannot be nested.

session.transaction(function(tx) {
  ...
});

Commit and Rollback

persistence.js works in autocommit mode by default.

You can override this behavior and enable explicit commit and rollback by passing true as first argument to persistence.transaction. You can then use the following two methods to control the transaction:

Typical code will look like:

session.transaction(true, function(tx) {
  // create/update/delete objects
  modifyThings(session, tx, function(err, result) {
    if (err) {
      // something went wrong
      tx.rollback(session, function() {
        console.log('changes have been rolled back: ' + ex.message);
      });
    }
    else {
      // success
      tx.commit(session, function() {
        console.log('changes have been committed: ' result);
    });
  });
});

Explicit commit and rollback is only supported on MySQL (server side) for now.

Defining your data model

Defining your data model is done in exactly the same way as regular persistence.js:

var Task = persistence.define('Task', {
  name: "TEXT",
  description: "TEXT",
  done: "BOOL"
});

A schemaSync is typically performed as follows:

session.schemaSync(tx, function() {
  ...
});

Creating and manipulating objects

Creating and manipulating objects is done much the same way as with regular persistence.js, except that in the entity's constructor you need to reference the Session again:

var t = new Task(session);
...
session.add(t);

session.flush(tx, function() {
  ...
});

Query collections

Query collections work the same way as in regular persistence.js with the exception of the Entity.all() method that now also requires a Session to be passed to it:

Task.all(session).filter('done', '=', true).list(tx, function(tasks) {
  ...
});

Closing the session

After usage, you need to close your session:

session.close();

Bugs and Contributions

If you find a bug, please report it. or fork the project, fix the problem and send me a pull request. For a list of planned features and open issues, have a look at the issue tracker.

For support and discussion, please join the persistence.js Google Group.

Thanks goes to the people listed in AUTHORS for their contributions.

If you use GWT (the Google Web Toolkit), be sure to have a look at Dennis Z. Jiang's GWT persistence.js wrapper

License

This work is licensed under the MIT license.

Support this work

You can support this project by flattering it: