Proposal - add subs.static property

[avalanche1]

I have a city picker in my proj with 5000 cities in it (each - a different doc). This picker is present on every page. Before using subs-manager I subscribed to 'cities' publication only when user clicked on city picker, showing a loader while the data was downloaded to the client. After installing subs-manager my data is there almost instantly (if user doesn't rush in and clicks city picker instantly after page was rendered). But this means that each of my users will be subscribed to this pub, inducing overhead on the server, related to maintaining all of these reactive subscriptions. So, I'm in doubts here:

• with subs-manager I get nice UX, but server overhead
• w\o it I subscribe only when necessary, but have suboptimal UX

Thus I propose to add to subs-manager the ability to download only the initial data and then sever connection to server if it doesn't need to be reactive. Like adding a subs.static: true property.

[mjmasn]

You could just create a meteor method to retrieve the cities and call it inside a Meteor.startup block on the client which would have the same effect. Save the cities to some kind of global variable like the Session var or a local collection var localCollection = new Mongo.Collection(null)?

[lassombra]

I agree with @mjmasn. This is exactly the wrong way to approach that problem. Either use a method and locally store it, or have a non-reactive publication (One that just fetches the records and dumps them using this.added in the publication method).

[avalanche1]

@lassombra how do I create a non-reactive publication?

[mjmasn]

@avalanche1 Another way is to do a null publish. The best way to explain it is with the Meteor.user() function. Where do your user details come from to allow the function to work? They are published using a null publish [1], which means the data is automatically sent to the client when the page first loads.

e.g. if your collection is called 'Cities':

Meteor.publish(null, function() { return Cities.find(); });

No need to subscribe, Meteor will handle it automatically because the publish has no name. Now you will always have access to the cities on the client without having to reload the subscription on each page. The list will still be reactive, but there's no overhead if nothing changes so you shouldn't need to worry about 'overloading' the server at all.

[1] https://github.com/meteor/meteor/blob/devel/packages/accounts-base/accounts_server.js#L699-L714

[lassombra]

@mjmasn this still has the problem of the entire subscription being reactive. What @avalanche1 is looking for is to not take that performance hit.

@avalanche1 to do a "non-reactive" publication which has the benefit of being tied to the mergebox still and still showing up as an active subscription, in your server code:

Meteor.publish('publicationName'/* or null for autopublish */, function() {
    // this in a publication context represents a specific publication.
    // save self to this to make it available in closures later.
    // (Or could use arrow functions with es6)
    var self = this;
    // get the entire current cities array.
    // At this point we're acting like a non-reactive controller
    var cities = Cities.find.fetch();
    // loop through the cities to send them to the client.
    _.each(cities, function(city) {
        // self.added from http://docs.meteor.com/#/full/publish_added
        // sends an item to the client to be stored in the client's
        // minimongo collection with the provided collection name.
        self.added('collectionName', city._id, city);
    });
    // don't forget to send this or the subscription will never know
    // that it's done and the data is ready to be used.
    this.ready();
});

The only real benefit this has compared to meteor methods is it handles the integration into minimongo for you both on the adding side, and on the removing side when you stop the subscription.

Otherwise you can have a meteor method that does something similar and you load it into minimongo yourself.

[avalanche1]

@lassombra , thank you, this is educative. But how can we stop the publication after receiving the initial set of data? I don't want the server to spend resources on maintaining that pub. ... Ah, I guess, we'd use this.stop() on the pub)

But anyway, I think it would be great to have a subs.static property in subs-manager instead of all this code above.

[lassombra]

@avalanche1

In the example I showed you, there is only a slight memory overhead on the server as it maintains the map of items sent so that on stop it can remove them. No observers are created which is where the real pain of publications exist. It can also be further optimized by having cities loaded into memory once on startup and always pumped down on request. If you want a method style:

//Server
Meteor.methods({'getCities': function() {
    return Cities.find().fetch();
}});

//Client
Meteor.startup(function(){
    Meteor.call('getCities', function(error, result){
        if (result){
            _.each(result, function(city){
                Cities.insert(city);
            });
        }
    })
});

This is the way to have the data stay in minimongo if you do not want to keep an active subscription.

But, as I said earlier, a non-reactive publication like I provided above is not expensive and you get minimongo integration automatically.

Back to subs.static, besides being outside of the scope of what subsmanager is, it's impractical. subs.static would never be useful here because it couldn't force the server to not be reactive. You'd still get an observer created (with all the cost there) and then quickly destroyed according to the design you offer which means then garbage collection. Once you call this.stop() the data is removed from local mongo. Subs manager would have to alter the core behavior of meteor in order to change that. This is not something we want. Also, you really don't want the client telling the server how to handle the publication. The code I provided above is really not that complicated, and if you're going to be doing it in multiple places, you could create a utility function to do it. If you want non-reactive data that you keep handy in local mongo, then you need a method. A method can retrieve information server side, and return it to the client which can then load it into either a local variable, or if needed, minimongo. A method once complete is done, there are no more references to it hanging around.

I've used the non-reactive publications more than a couple times on production applications which had just way too much data. In one case, the publication is cached so that every client that ever connects gets the same result until the server process is restarted. In another case the publication is refetched every time but doesn't react so I don't incur that overhead.