search

robfallows / tunguska-reactive-aggregate

Reworks jcbernack:reactive-aggregate for ES6/ES7 and Promises
MIT License
51 stars 27 forks source link

readme

tunguska-reactive-aggregate

Versions and recent changes

Meteor 3.x

Use tunguska-reactive-aggregate v2.0.1

Meteor 2.x

Use tunguska-reactive-aggregate v1.3.16

About

Reactively publish aggregations.

Originally based on jcbernack:reactive-aggregate.

This version removes the dependency on meteorhacks:reactive-aggregate and instead uses the underlying MongoDB Nodejs library. In addition, it uses ES6/7 coding, including Promises and import/export syntax, so should be imported into your (server) codebase where it's needed.

In spite of those changes, the API is basically unchanged and is backwards compatible, as far as I know. However, there are several additional properties of the options parameter. See the notes in the Usage section.

Changed behaviour in v1.2.3: See https://github.com/robfallows/tunguska-reactive-aggregate/issues/23 for more information.

History

See changelog.

meteor add tunguska:reactive-aggregate

This helper can be used to reactively publish the results of an aggregation.

Mongo.ObjectID support

If your collections use the Meteor default of String for MongoDB document ids, you can skip this section and may want to set options.specificWarnings.objectId = false and options.loadObjectIdModules = false

However, if you use the Mongo.ObjectID type for document ids, full support for handling Mongo.ObjectIDs is only enabled if simpl-schema and either lodash-es or lodash are installed. For backward compatibility, they are not required. (Only the set functionality of lodash-es/lodash is imported, if you're concerned about the full package bloating your code size).

You can install them in your project with:

meteor npm i simpl-schema

meteor npm i lodash-es or meteor npm i lodash

Additionally, unless you have defined SimpleSchemas for your collections, you still won't have full support for handling Mongo.ObjectIDs. The _id field of your primary collection will be handled properly without installing these packages and without having SimpleSchemas defined, but any embedded Mongo.ObjectID fields will not be handled properly unless you set up full support with these packages and schema definitions. Defining SimpleSchemas is beyond the scope of this writeup, but you can learn about it at simple-schema on GitHub.

If you're curious about why Mongo.ObjectIDs require special support at all, it's because in Meteor, aggregate must use the low-level MongoDB Nodejs library, which doesn't know the Mongo.ObjectID type and so performs conversions that break Mongo.ObjectIDs. That's what 'full support' here is working around.

Usage

import { ReactiveAggregate } from 'meteor/tunguska:reactive-aggregate';

Meteor.publish('nameOfPublication', function() {
  ReactiveAggregate(context, collection, pipeline, options);
});

Quick Example

A publication for one of the examples in the MongoDB docs would look like this:

Meteor.publish("booksByAuthor", function () {
  ReactiveAggregate(this, Books, [{
    $group: {
      _id: "$author",
      books: { $push: "$$ROOT" }
    }
  }]);
});

Extended Example

Define the parent collection you want to run an aggregation on. Let's say:

import { Mongo } from 'meteor/mongo';
export const Reports = new Mongo.Collection('Reports');

...in a location where all your other collections are defined, say /imports/both/Reports.js

Next, prepare to publish the aggregation on the Reports collection into another client-side-only collection we'll call clientReport.

Create the clientReport in the client (it's needed only for client use). This collection will be the destination into which the aggregation will be put upon completion.

Publish the aggregation on the server:

Meteor.publish("reportTotals", function() {
  ReactiveAggregate(this, Reports, [{
    // assuming our Reports collection have the fields: hours, books
    $group: {
      '_id': this.userId,
      'hours': {
      // In this case, we're running summation.
        $sum: '$hours'
      },
      'books': {
        $sum: 'books'
      }
    }
  }, {
    $project: {
      // an id can be added here, but when omitted,
      // it is created automatically on the fly for you
      hours: '$hours',
      books: '$books'
    } // Send the aggregation to the 'clientReport' collection available for client use by using the clientCollection property of options.
  }], { clientCollection: 'clientReport' });
});

Subscribe to the above publication on the client:

import { Mongo } from 'meteor/mongo';

// Define a named, client-only collection, matching the publication's clientCollection.
const clientReport = new Mongo.Collection('clientReport');

Template.statsBrief.onCreated(function() {
  // subscribe to the aggregation
  this.subscribe('reportTotals');

// Then in our Template helper:

Template.statsBrief.helpers({
  reportTotals() {
    return clientReport.find();
  },
});

Finally, in your template:

{{#each report in reportTotals}}
  <div>Total Hours: {{report.hours}}</div>
  <div>Total Books: {{report.books}}</div>
{{/each}}

Your aggregated values will therefore be available in the client and behave reactively just as you'd expect.

Using $lookup

The use of $lookup in an aggregation pipeline introduces the eventuality that the aggregation pipeline will need to re-run when any or all of the collections involved in the aggregation change.

By default, only the base collection is observed for changes. However, it's possible to specify an arbitrary number of observers on disparate collections. In fact, it's possible to observe a collection which is not part of the aggregation pipeline to trigger a re-run of the aggregation. This introduces some interesting approaches towards optimising "heavy" pipelines on very active collections (although perhaps you shouldn't be doing that in the first place :wink:).

Meteor.publish("biographiesByWelshAuthors", function () {
  ReactiveAggregate(this, Authors, [{
    $lookup: {
      from: "books",
      localField: "_id",
      foreignField: "author_id",
      as: "author_books"
    }
  }], {
    noAutomaticObserver: true,
    debounceCount: 100,
    debounceDelay: 100,
    observers: [
      Authors.find({ nationality: 'welsh'}),
      Books.find({ category: 'biography' })
    ]
  });
});

The aggregation will re-run whenever there is a change to the "welsh" authors in the authors collection or if there is a change to the biographies in the books collection.

The debounce parameters were specified, so any changes will only be made available to the client when 100 changes have been seen across both collections (in total), or after 100ms, whichever occurs first.

Non-Reactive Aggregations

Like a Meteor Method, but the results come back in a Minimongo collection.

Meteor.publish("biographiesByWelshAuthors", function () {
  ReactiveAggregate(this, Authors, [{
    $lookup: {
      from: "books",
      localField: "_id",
      foreignField: "author_id",
      as: "author_books"
    }
  }], {
    noAutomaticObserver: true
  });
});

No observers were specified and noAutomaticObserver was enabled, so the publication runs once only.

On-Demand Aggregations

Also like a Meteor Method, but the results come back in a Minimongo collection and re-running of the aggregation can be triggered by observing an arbitrary, independent collection.

Meteor.publish("biographiesByWelshAuthors", function () {
  ReactiveAggregate(this, Authors, [{
    $lookup: {
      from: "books",
      localField: "_id",
      foreignField: "author_id",
      as: "author_books"
    }
  }], {
    noAutomaticObserver: true,
    observers: [
      Reruns.find({ _id: 'welshbiographies' })
    ]
  });
});

By mutating the Reruns collection on a specific _id we cause the aggregation to re-run. The mutation could be done using a Meteor Method, or using Meteor's pub/sub.

Enjoy aggregating reactively, but use sparingly. Remember, with great reactivity comes great responsibility!