Unblock doesn't work when publishing a large collection

[bradvogel]

I'm publishing a large collection and using this.unblock(). However, I'm seeing that the queue is still still blocked as the collection is being streamed back. Here is my test case:

On the server:

Meteor.publish('largeCollection', function() {
  this.unblock();
  return LargeCollection.find();
});
Meteor.methods({
  ping: function() {
    return '';
  }
});

On the client (works correctly):

var start = Date.now();
Meteor.subscribe('largeCollection', function(){
    console.log('largeCollection finished after ', Date.now() - start, 'ms');
});
// Called immediately after, this should return first.
Meteor.call('ping', function(e,r) {
   console.log('ping returned after ', Date.now() - start, 'ms');
});

This correctly returns:

ping returned after  340 ms
largeCollection finished after  17380 ms

However, if the ping method gets called a bit later, it DOES NOT return immediately. Instead, it returns after the collection is done streaming:

On the client (does not work as expected):

var start = Date.now();
Meteor.subscribe('largeCollection', function(){
    console.log('largeCollection finished after ', Date.now() - start, 'ms');
});
// Delay the call for a second.
setTimeout(function(){
    Meteor.call('ping', function(e,r) {
       console.log('ping returned after ', Date.now() - start, 'ms');
    });
}, 1000);

This returns:

largeCollection finished after  16567 ms
ping returned after 17267 ms

So in the second case where the call is delayed, it seems to wait until after the collection is done streaming. Is this expected? Shouldn't this.unblock() mean that all calls after should go through the queue?

[arunoda]

I think this is expected. When sending a large collection data, it's a node process blocking operation and that's why you got this issue.

unblock can't help on these situations.

---

Arunoda Susiripala I curate Meteor Weekly - Check it out! http://meteorhacks.com/meteor-weekly/?utm_source=email-footer&utm_medium=email&utm_campaign=meteorweekly

On Wed, Oct 1, 2014 at 2:53 AM, Brad Vogel notifications@github.com wrote:

I'm publishing a large collection and using this.unblock(). However, I'm seeing that the queue is still still blocked as the collection is being streamed back. Here is my test case:

On the server:

Meteor.publish('largeCollection', function() { this.unblock(); return LargeCollection.find(); }); Meteor.methods({ ping: function() { return ''; } });

On the client (works correctly):

var start = Date.now(); Meteor.subscribe('largeCollection', function(){ console.log('largeCollection finished after ', Date.now() - start, 'ms'); }); // Called immediately after, this should return first. Meteor.call('ping', function(e,r) { console.log('ping returned after ', Date.now() - start, 'ms'); });

This correctly returns:

ping returned after 340 ms largeCollection finished after 17380 ms

However, if the ping method gets called a bit later, it DOES NOT return immediately. Instead, it returns after the collection is done streaming:

On the client (does not work as expected):

var start = Date.now(); Meteor.subscribe('largeCollection', function(){ console.log('largeCollection finished after ', Date.now() - start, 'ms'); }); // Delay the call for a second. setTimeout(function(){ Meteor.call('ping', function(e,r) { console.log('ping returned after ', Date.now() - start, 'ms'); }); }, 1000);

This returns:

largeCollection finished after 16567 ms ping returned after 17267 ms

So in the second case where the call is delayed, it seems to wait until after the collection is done streaming. Is this expected? Shouldn't this.unblock() mean that all calls after should go through the queue?

— Reply to this email directly or view it on GitHub https://github.com/meteorhacks/unblock/issues/1.

[bradvogel]

Ok, makes sense, but I'd think it would be I/O bound and not CPU bound?

[arunoda]

Hmm. I think that's CPU bound.

we need to check it properly. If that's IO bound things should work. Once meteor detects changes it will send all the data and that's a CPU blocking operation.

But I suspect that 17 secs is a huge number. Can't say much more without a test case (I mean a sample app)

---

Arunoda Susiripala

@arunoda http://twitter.com/arunoda http://gplus.to/arunodahttps://github.com/arunoda http://www.linkedin.com/in/arunoda

On Wed, Oct 1, 2014 at 7:16 AM, Brad Vogel notifications@github.com wrote:

Ok, makes sense, but I'd think it would be I/O bound and not CPU bound?

— Reply to this email directly or view it on GitHub https://github.com/meteorhacks/unblock/issues/1#issuecomment-57408789.

[arunoda]

I think we can close this issue?