Realtime handling

[meaku]

The current concept is not perfect and lacks flexibility. I'm thinking of making it easier and more pub-sub-link.

Ideas

https://github.com/LearnBoost/socket.io/wiki/Rooms

PubSub

socket.on('subscribe', function(data) { socket.join(data.room); })
socket.on('unsubscribe', function(data) { socket.leave(data.room); })

The application itself needs some logic to decide if clients are allowed to subscribe a certain event_name / room. Maybe we could provide special services for pub/sub.

Emitting

io.sockets.in('room').emit('event_name', data)

The emitting should happen in a middleware or a service. It should emit the data to all clients that have subscribed event_name.

Automatic pub/sub

alamid is automatically pub/sub'ing in order to automatically CRUD models. I would like to keep this handling, but make it more configurable by overwriting or extending the behavior.

[jhnns]

In terms of security and performance we should not pub/sub automatically. The developer should enable this feature manually so he indicates that he knows what he's doing. I think this feature could make the application unintended slow and insecure because everybody can subscribe to events that he might not know about.

[meaku]

That's part of what i meant with:

but make it more configurable by overwriting or extending the behavior.

[meaku]

But you are right. I could have written it down :)

[jhnns]

We should do some research on this. I think the Meteor Documentation could provide some useful hints.

I'm trying to sort my own thoughts on this topic by writing them down.

Overview

We need 4 components to make this feature work:

• A mechanism to subscribe to a specific collection of models
• A mechanism to publish to these subscribers
• A mongoose-like filter that selects the subscribers when publish() is called
• An API for external requests to subscribe to specific model type

Pub/sub is almost the same like emitting events. In alamid there are two differences:

• With pub/sub usually you want to pass a filter that is run before the listener is actually called
• In alamid we have to differentiate between remote and local. This is important for performance reasons.

So on the client there is:

• Local subscribing
• Remote subscribing
• Local publishing

There is no remote publishing on the client because only the server notifies its clients. Another problem on the client is the fact, that we usually don't want to be notified about "remote changes" that have been triggered by the own client.

On the server there is:

• Local subscribing
• Local publishing
• Remote publishing

There is no remote subscribing on the server, because the request is done explicitly by the client.

Currently local publishing is always done automatically by calling model.save(). But I think we should keep it consistent and put it all in the hands of the developer. On the client local publishing is usually not a problem, but it is on the server.

To differentiate between these two mechanisms - pub/sub and events - we should replace all static EventEmitter functions with pub/sub-functions. on() becomes subscribe(), emit() becomes publish(). I think it gets confusing if the model class would provide both interfaces.

Model instances on the other hand still provide the EventEmitter interface. So you could say: Events notify about a state change of a specific instance, subscriptions notify about a state change in a collection.

Subscribing

So if you're subscribing, you basically need to pass 3 things:

• The model class you want to be informed about
• The event types you want to be informed about, typically create, update and delete. read is an edge-case, but maybe there are some situations where you need it so alamid should also provide it.
• Filter parameters on the model, like { author: "siR", creationDate: 1039853284 }. Since alamid provides also the possibility to have embedded models like blog/post/comment we have to differentiate here between ids and arbitrary filter parameters.

On the client you need also to pass the source, e.g. local or remote. I'm not sure about this local and remote thing yet but a possible implementation would look like:

Post.server.subscribe("create", params, function onPostCreate(newPost) { ... });

// Not available on the server
Post.client.subscribe("create", params, function onPostCreate(newPost) { ... });

// Subcribes client and server
Post.subscribe("create", params, function onPostCreate(newPost) { ... });

params is an mongoose-like filter object like you would pass it to Model.find(). We have to sort out whether ids should be passed via this object or not.

Maybe it's not good to aggregate it all into the Model-Class. We could also make this part of the ModelCollection since subscription is always related to a collection of models.

Publishing

Publishing is easy

Post.client.publish("create", post);

// Not available on the client
Post.server.publish("create", post);

// Publishes on the client and server
Post.publish("create", post);

API for external subscription requests

I think we should utilize alamid's route handler (node middler respectively :wink:) for this subscription task because it is basically similar to handling CRUD requests. It's very common to use the same middleware for many different models. The only difference is that these requests are not REST-ful. It's a contract between the server and another client. So the router handler needs the possibility to access the client when a specific event occurred.

It is also necessary to automatically unsubscribe clients if they disconnect or haven't been active for a while (I'm not sure if alamid should handle the latter).

I'm not sure about the alamid API for this feature. This text is way to long, I need a break :bread: :beers:

[jhnns]

I've thought about this issue again.

I would definitely remove the EventEmitter from the static Model. Combining both concepts in one class is confusing. But we could describe pub/sub as possibility to emit an event from the server to the client. So the EventEmitter is for communication between objects, pub/sub for communication across the network.

The server can also subscribe, but I would remove the feature to define the scope when publishing. So publishing will always inform all subscribed clients, too.

// Subscribing on the client and server
Post.subscribe("create", params, function onPostCreate() { ... });

// Only available on the server
Post.publish("create", post);

[jhnns]

I've just had the idea of view-piping :laughing:. Just imagine how awesome this interface would be:

Post.pipe(params, viewCollection);

This subscribes create, update, destroy to the given channel and binds the resulting model collection to the view collection.

[meaku]

pipe pipe pipe pipe pipe! awesome! collections should be streams! that's a great idea!

Michael Jaser Gesendet mit Sparrow (http://www.sparrowmailapp.com/?sig)

Am Dienstag, 11. Dezember 2012 um 01:25 schrieb Johannes:

I've just had the idea of view-piping . Just imagine how awesome this interface would be: Post.pipe(params, viewCollection);

This subscribes create, update, destroy to the given channel and binds the resulting model collection to the view collection.

— Reply to this email directly or view it on GitHub (https://github.com/peerigon/alamid/issues/129#issuecomment-11225557).

[jhnns]

P-p-p-p-ipe

[meaku]

The main problem is how define which users should be notified by "publish". In a simple pub sub, you would notify all users that have subscribed. But in most real world-applications you usually need some kind of authentication and only certain users should be able to subscribe to certain models. And in my use-case only the publish-server-code knows who the messages should be delivered to...

[jhnns]

Authentication is handled in meteor.js within the event listener for the subscribe event.

[meaku]

I did some prototyping and came up with this - pseudo-code - draft:

Server

ModelPubSub.server.js

Model.remotePublish = function(method, model, receivers) {

    var ws = websocket.getInstance();

    if(receivers === undefined) {
        receivers = [""]; //global room as fallback
    }

    receivers.forEach(function(receiver) {
            //websocket
            if(io.sockets.manager.rooms["/" + receiver] !== undefined) {
                io.sockets.in(receiver).emit("remotePublish", method, model.toObject());
            }
    });
}

Model.publish(method, model) {
    this.emit.call(arguments);
}

Model.subscribe = function(method, subscribeHandler) {
    this.on.call(arguments);
}

Client

ModelPubSub.client.js

//must only be called once on init
Model.enableRemoteSubscribe = function() {

    var self = this;

    io.on("remotePublish", function(method, modelData) {

        //add to model cache 
        //or do update if existing
        var model = new Model(modelData.id);
        model.set(modelData);

        self.publish(method, model);
    });
}

Model.subscribe = function(method, subscribeHandler) {
    this.on.call(arguments);
}

Model.publish = function(method, model) {
    this.emit.call(arguments);
}

I would keep the API consistent and offer publish and subscribe on client & server. But as you proposed i would not publish automatically anymore. And the old events will be replaced with "publish" and "subscribe".

The implementation on client & server is almost the same.

RemotePublish is a special case, which has to be enabled on the client, by calling enableRemoteSubscribe. In order to publish to the client you have to call remotePublish instead of publish. Another option would be to give publish an additional attribute called "remote", which can be set to true if you want to remotePublish.

And i'm not sure if we should add a attribute which indicates if the publish happened local or remote on the client?

Should the ModelPubSub be implement as a mixin/plugin or should it be backed in the model?

What do you think @jhnns?

[jhnns]

Mhmm ... I think we should first try to take the user's perspective. So we should try to use a mock-API in peerigon and we'll see if it's cool.

The PubSub-thing should definitly be an own mixin.