Feathers 1.1.0 roadmap and todos

[daffl]

After merging @marshallswain's great #107 PR we are on the road for a version 1.1.0. Here is some things I'd like to discuss and do:

Messaging

A service oriented architecture make things a lot easier. That's why Feathers exists. I am aware that this also needs to be communicated more clearly but Microservices is the buzzword that sums it up pretty well. Should it be lined up as a microservice library on top of Express? Is the reference to Express even necessary?

Homepage

• [x] Freshen up the homepage, move the guide into its own page
• [ ] Feature the Feathers talk from https://www.youtube.com/watch?v=2FtLRDX3yIc

  Features

• [ ] Research and potentially add support for ZeroRPC to more easily communicate with distributed services

  Plugins

• [ ] Create a feathers-authentication plugin that wraps feathers-passport and makes it easier to use (almost done)
• [x] Finish the feathers-websocket-client and use it for client side real-time integration (including canjs-feathers)
• [x] With which it should also be very easy adding MV* framework adapters. I volunteer @ekryski for React and Angular

Let me know your thoughts /cc @ekryski, @marshallswain, @Glavin001

[marshallswain]

Freshen up the homepage

Are you thinking extreme makeover, new design fresh?

Support for ZeroRPC

Sounds so good.

Is the reference to Express even necessary?

I say yes, until we support custom methods in services. Custom methods have to be added in each provider right now. Once it is done, I don't think it will matter that we mention Express at all. The ability for services to register custom methods is something I would really like to see added to this list. But that might be a bit much for 1.1.0...?

In light of @daffl 's comments about sticking to REST, below, we don't need to mention Express when we stick to RESTful services.

[marshallswain]

Create a feathers-authentication plugin

I am really looking forward to seeing this. I made my first shot at a plugin with feathers-accounts. I was originally going to make a token-based (no cookies) auth provider for feathers-passport (which I still plan on doing to eliminate CSRF attacks), but decided to explore the limits of plugin creation. I think some custom-method love would make creating powerful plugins really easy.

The goal with feathers-accounts was to make it as easy as app.configure(accounts(config)) to add user signup, login, and pluggable messaging for signup and "forgot password" functions on the server, then make client modules to make it super easy to implement in the browser.

I'm probably reinventing the wheel, since you're already working on an auth plugin, but I really want to help make something as powerful as meteor-accounts and meteor-accounts-ui. However we end up doing it, it would be a really good idea to make token-based auth the encouraged standard. I'd like to even add support for two-factor auth using Twilio messaging or email-to-text for low-traffic apps.

[marshallswain]

I looked a little more into ZeroRPC. It looks like the protocol that it runs on doesn't have any built-in security at the moment, so if we want to secure communications at the server level we would have to implement our own layer on top of it. They do have a spec for the next version of the protocol. A flexible security implementation seems to be the main focus.

I don't think this should deter us from adding support, though. The clear-text messaging can be secured outside the Feathers server. We will just want to be clear about this in the docs.

[ekryski]

I like that plan @daffl! I can help give feathers a bit of a facelift. I feel like I've improved my design skills a bunch since we last looked at things.

I also :heart: the idea of pushing towards micro services as that was I had initially in mind for feathers when we first started it. I'll try and weigh-in on things but I may not have a lot of time until I head on vacation in a few days.

[ekryski]

And yes I can do react and angular plugins. React might be first. Thy are doing some really exciting things over there.

[ekryski]

I also second @marshallswain's suggestion for registering custom services soon. Will make things much cleaner for people to grok how things are built "the feathers way" since routes/services will be more consistent with how they are created.

[daffl]

I'm not sure what you guys mean with custom service methods. There is a very specific reason why services only provides a REST architecture pattern compliant interface (which I should probably write up soon). Where would something like { sayHello: function(name, callback) {} } set a URL route to (it's definitely not myservice/sayHello because that's everything that's wrong with borked "RESTful" applications)?

I wrote my thesis paper about this in University and I have been using that pattern and interface even before Feathers for years now in many applications and never once ran into a case that couldn't be reduced into a basic CRUD functionality (which translates right into a RESTful API). Much like routing I think this is something that's being horribly overcomplicated and overrated. I really want to save yet another engineer convincing themselves that "My application is more complicated than CRUD" - because it simply isn't and there is no reason giving up a sound architectural approach because of that assumption.

Unfortunately it seems to be generally underrated (or unknown) how much thought and experience went into REST as an architectural design pattern (with which I mean the original PHD thesis chapter you can read at http://www.ics.uci.edu/~fielding/pubs/dissertation/rest_arch_style.htm). When one of the people that brought us the Internet™ sits down and distills the design decisions that made the biggest distributed system ever possible and reliable into an architecture design pattern it is not something that should be easily dismissed and I'm even hoping to make it more accessible.

Anyway, </rant> end :stuck_out_tongue_winking_eye: I'm not saying I'm not open to the option at all, I just feel that it would only treat the symptoms instead of the problem.

[daffl]

Oh and please correct me if I misunderstood this. Broken APIs are one of my pet peeves (since I'm one of the people that have to work with them every day ;) so I want to make it as hard as possible to create one. Maybe we should create new separate issues and discuss it there with an example?

[marshallswain]

Hmm. Ok good point. I really don't have a strong use case for not using REST. I was thinking that user signup and login would handled inside one service, but it can be easily spread across multiple services. POST to /users for signup. POST to /passwordresets to start that process. POST to /tokens to trade login details for a token. I just needed to adjust my thinking a bit. :)

[ekryski]

@daffl No I get where you are coming from. I feel like I recently had this "ah ha!" moment, with the latest app good.pn. We actually don't need deeply nested routes. The one thing I do find myself doing right now is creating a service called "public" which houses my public/static page routes. For example:

module.exports = function() {
  var app = this;

  app.get('/', function(req, res, next){
    res.render('static/home', {});
    // res.redirect('/live');
  });

  app.get('/faq', function (req, res) {
      res.render('static/faq', {});
  });
};

These routes obviously don't really benefit from using feathers. However, it does get more complicated when you are dealing with routes like password reset. In that case you may want to use hooks, you may want socket support, and you will need to talk to a datastore but it's not really a resource. You're not going to have a GET on /passwords.

After writing this out, maybe it makes more sense for us to set up a series of best practices for what non-service routes (ie. routes not hooked to a resource) should look like. Curious to hear thoughts there.

[marshallswain]

Does RESTful mean it has to implement all of the service/resource methods?

[daffl]

No, you can always say that a methods hasn't been implemented. And you're totally right, it needs a little change in thinking which is why I want to write more posts about those topics. Just releasing a framework is definitely not enough.

On my walk to the office I was thinking about your points though and I think there is a case when you might need custom events. REST is stateless by definition so it doesn't incorporate everything you can do with websockets, for example notifying a user about a payment process. I think we might be able to easily add this though because services are already event emitters. Lets consider a payment service that sends status information to the user while the payment is processed (pardon my ES6, I hope it makes sense anyway ;):

var paymentService = {
  events: [ 'log' ],

  create(data, params) {
    var log = message => this.emit('log', { message: message, user: params.user });
    var errorHandler = error => log(`There was an error processing your payment: ${error.message}`);

    return addCreditCard(data.cc).then(cc => {
      log(`Credit card ${cc.displayName} added successfully`);

      return processPayment(data).then(paymentInfo => {
        log(`Payment of ${paymentInfo.amount}.- successful!`);

        return paymentInfo
      });
    }).fail(errorHandler);
  }
};

app.use('/payments', paymentService);

And on the client we could do:

socket.on('payments log', function(data) {
  $('#payment .logs').append(`<li>${data.message}</li>`);
});

[marshallswain]

So a blog ought to be in the home page refresh, or at least a feed of Feathers articles, since Medium is probably where y'all's articles will end up, right?

I'd like to do a multi-part series of tutorial posts on building a useful demo application. I'm working on the MongoDB manager that I mentioned a while back. I was going to do that as the tutorial app, but building an app that uses dynamic services to create other dynamic services might hurt a beginner's head. I'd like to do something that would be useful to a bigger audience. Any ideas?

Maybe the world could use another CMS, only based on Feathers. :)

[marshallswain]

And I definitely can see a place for custom events.

[daffl]

A blog maybe. What do you guys think would be better? Writing on Medium and starting a collection or putting up blog posts on the website? I found Medium (which now allows to embed Gists by the way) gives better exposure is easier to share and encourages me to produce higher quality content (which is also the downside since I'm for some reason uncomfortable just throwing some small tidbits on there). Is it maybe possible to write on Medium through a shared feathersjs Twitter account for things like release notes or overview of a new plugin?

Either way I love @marshallswain's idea for a bigger application walkthrough. I find it pretty hard to come up with a good application that is complex enough to show everything yet trivial enough to be able to follow along and that shows where the real-time components come in handy.

To follow up on what @ekryski said (since I didn't read the last part before ;), no you don't do GETon /passwords but it's still a resource that you create and update. I however usually just set up standard Express POST and GET routes for resetting a password or creating an account that uses the user service to create the account or update the password. I can't think of a non-service route that needs to be available through all providers. I actually have an article for that in the works.

[marshallswain]

How about using Medium Embeds, which would end up looking something like this:

[daffl]

I think we can even create a Feathers specific publication and embed that one. Oh also, I just joined https://gitter.im/feathersjs, might be easier for chatting.

[marshallswain]

We should put logos of the frameworks for which we have custom adapters on the homepage. Give people some feel-goods about it working with their framework.

[daffl]

Absolutely. Maybe have "Real-time with..." and then the logos. Now we just need the adapters ;)

[marshallswain]

Yeah, but the CanJS logo will be pretty all by itself, still.

[agonbina]

One thing I would like to be able to do is use a service without exposing it as a REST or Socket.io "route".

example:

var service = memory()
var app = feathers().use('/todos', service)

module.exports = app.service('todos')

somewhere else using this service:

var todos = require('./service')

todos.on('created', sendToMessageBus)
todos.create({ ... })

Some of this functionality works already, but things like listening for events do not make the Node process stay running, or not passing a callback to .create throws an error.

I think being able to use services this way, we can treat them as an abstraction between our data layer(which can be swapped without changing how the service behaves) and the logic that runs before/in/after the service method, without having to expose them publicly, staying close to the microservices concept. You could choose to expose specific services via different apps(node processes), but not all of them. Or you can compose smaller apps by mixing and matching different services and feathers plugins.

I am no "microservices" expert, but code wise this would work really nice for my use cases. I haven't seen any other Node.js library that can do this as cleanly as a feathers service, but if I am moving completely away from what Feathers goals are, or if I'm missing something, please correct me :+1:

[daffl]

No, that does make sense. Most of this should already work although it seems that calling methods with different parameters is something that should really be supported. I'm not sure what you mean that listening to evens does not keep the Node process running (I know that events are not published across different processes).

I was envisioning to use ZeroRPC for inter-process communication (which should also publish events across all clients) which should allow to do this:

var feathers = require('feathers');
var zeroRPC = require('feathers-zerorpc');
var memory = require('feathers-memory');

var type = process.env.TYPE;
var app = feathers();

app.configure(zeroRPC());

if(type === 'USERS') {
  app.use('/users', memory())
} else {
  app.use('/users', zeroRPC.service({
    host: 'tcp://localhost:2447'
  }));
  app.service('users').on('created', function(user) {
    console.log('New user created on host:', user);
  });
}

You can scale this over as many processes on one machine or as many machines on the same network as you want which is really nice.

[agonbina]

By keep the Node process running I mean when you say service.on('created', callback) even if you are not running an HTTP server, the process will keep running and trigger the callback of the event on service.create() calls.

I haven't looked much into zeroRPC but I prefer to use a message queue, so the current Feathers architecture works out for building microservices that are exposing endpoints publicly. Sorting out those little things like allowing calls to all service methods w/o requiring all arguments to be passed in, and maybe running some of them automatically(ex. service.setup) would make Feathers even more suitable for running non HTTP Node processes(in my case at least).

[daffl]

That is definitely a great discussion to have for version 2. The Express compatibility (and must-have HTTP server) was mainly because REST and websockets were the first providers. It would indeed be much better to have the main application be a service registry that you can plug providers into. For now I guess starting an HTTP server that does nothing works but it isn't as nice.

I guess we were looking at ZeroRPC because it seems very easy to implement. I'd love to add support for ZeroMQ and Redis pub/sub as well. The only stipulation is that just like websockets the message queue needs to allow a callback so we can do something like:

socket.pub('todos::create', JSON.stringify(data), function(error, response) {

});

Unfortunately I am not too familiar with any of those messaging services yet but having a publish message acknowledgement mechanism available doesn't seem too far fetched.

[daffl]

We can probably close this since 1.1 and a pretty new website is out. 1.2 (or 2.0) other features will be added as we go.

[lock[bot]]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue with a link to this issue for related bugs.