Cover use cases for node in a better way

[thecolorblue]

In my experience, there are three patterns that my code uses:

• express middleware
• Promise's
• Streams

Some ideas for how each of these might work:

Express Middleware

// would add product to `req.product` for every route with a `product_id` parameter
app.param('product_id', Moltin.express.getProductById); 

app.use(Moltin.express.proxy); // proxy requests to Moltin API

Promise's

Basic idea here is that requests return a promise object instead of requiring a callback. Personally, I like this style better than callbacks, but I think it would be better to make this an option. If this is not part of the library, look into testing Bluebird.js promisify functionality to get this.

Instead of:

moltin.Authenticate(function() {
  moltin.Category.List(null, function(categories) {
    moltin.categories = categories.result;
    console.log('moltin ready');
  }, function(error) {
      // handle category request error
  });
}, function() {
  // handle auth error here
});

... it would look like:

moltin.Authenticate()
  .then(function() {
    return moltin.Category.List(null);
  })
  .then(function(categories) { moltin.categories = categories; })
  .catch(function() { /* handle all errors */ });

Streams

I think streams are the killer feature for node.js. I do find them more useful when working with binary files, but they still apply here. When moving a lot of data around, a stream will be much more efficient and could lead to a lot of interesting use cases. This could also make it easier to use moltin with grunt, gulp, or other command line tools.

/* simple example */
var stream = moltin.stream.Category.List();

stream.pipe(httpResponse);

/* more involved */
var stream = require('stream');
var util = require('util');
var mongo = require('mongodb');
var mongoWriteStream = require('mongo-write-stream');
var collectionWriteStream = mongoWriteStream('mongodb://hostname/mydb');

function ConvertToMongo() {
  // init Transform
  Transform.call(this, {});
}
util.inherits(ConvertToMongo, Transform);
ConvertToMongo.prototype._transform = function (product, encoding, done) {
  product._id = mongo.ObjectId(product._id);
  // do other custom stuff here

  this.push(product);
  done();
};

// try it out
var readStream = moltin.stream.Product.List();
var convert = new ConvertToMongo();

// this could also be a http response, which would be more efficient for a long list of results
var writeStream = collectionWriteStream('products'); 

readStream
  .pipe(convert)
  .pipe(writeStream);

[sethkrasnianski]

Definitely :+1: for using catch with promises. It's definitely the defacto way to use promises and relatively easy to implement with bluebird or another well known lib. However, some folks truly enjoy the callback method of node, considering it more functional. Either way, offering both for each function in the library would probably best.

Middleware and streams may be more appropriate for a different repository since they could internally leverage this library under the hood and separate concerns. Breaking middleware and streams out would prevent a monolithic library such as express was a few major versions back.

I'm very interesting in hearing a few of your potential use cases for streams, particularly with the build tools. A quick high level explanation would do, but if you want to offer more pseudo or real examples then :smile: :smile:

[thecolorblue]

In build tools, streams might only be relevant in gulp.js. I do not know enough about grunt plugins to know if they would work.

Basically, I was thinking it would be nice to back up a store, download all of its products/brands/collections, and save them to a json file. It would also be good to restore a store from a backup or create a new store from a backup. It is cumbersome maintaining a production and a development environment with moltin, and this would help. Streams would not necessarily make this possible, but would make this more efficient.

[thecolorblue]

With stream support, we could tap into a whole collection of libraries like highland.js that would make working with data easier. Right now, the library cannot plug into other tools very well.

[artsmc]

This looks good so far. I don't claim to be a expert haha but I like idea of using streams to make the exporting and importing more efficient. We have built this for our projects because it was such a pain moving our set up between clients. Even for a simple store we have to manage 30+ products when we account for variations