percolatestudio / publish-counts

Meteor package to help you publish the count of a cursor, in real time
https://atmospherejs.com/tmeasday/publish-counts
MIT License
200 stars 46 forks source link

Counts.get() delivers zero even though MongoDb console and server show count > 0 #75

Open a4xrbj1 opened 9 years ago

a4xrbj1 commented 9 years ago

Autopublish package is removed. Here's the code in counts.html:

if (Meteor.isServer) {
    Meteor.publish('jobCompleted', function() {
        Counts.publish(this, 'numJobCompleted', myJobs.find({status: {$eq: "completed"}}));
    });

    Meteor.publish('atDNA', function() {
        Counts.publish(this, 'numAtDNA', Gedmatches.find({ atTotalCm: {$exists:true} }));
    });
}

if (Meteor.isClient) {
    Meteor.subscribe('jobCompleted');
     console.log('Number of jobs completed (client): ', Counts.get('numJobCompleted'));

    Meteor.subscribe('atDNA');
    console.log('Number of atDNA matches (client): ', Counts.get('numAtDNA'));
}

Output in the browser console is:

Number of jobs completed (client): 0 Number of atDNA matches (client): 0

Whereas the same query:

console.log('Number of jobs completed (server): ' + myJobs.find({status: {$eq: "completed"}}).count());
console.log('Number of atDNA matches (server): ' + Gedmatches.find({ atTotalCm: {$exists:true} }).count());

executed on the server (and in Mongodb shell) gives:

Number of jobs completed (server): 207 Number of atDNA matches (server): 51

I'm not sure what I'm doing wrong, tried to copy everything from the documentation. FYI - A normal find with a pub for all records (without further filter) find all records and works.

Thanks in advance for your help!

boxofrox commented 9 years ago

Try this example

if (Meteor.isServer) {
    Meteor.publish('jobCompleted', function() {
        Counts.publish(this, 'numJobCompleted', myJobs.find({status: {$eq: "completed"}}));
    });

    Meteor.publish('atDNA', function() {
        Counts.publish(this, 'numAtDNA', Gedmatches.find({ atTotalCm: {$exists:true} }));
    });
}

if (Meteor.isClient) {
    Meteor.subscribe('jobCompleted', function () {
        console.log('Number of jobs completed (client): ', Counts.get('numJobCompleted'));
    });

    Meteor.subscribe('atDNA', function () {
        console.log('Number of atDNA matches (client): ', Counts.get('numAtDNA'));
    });
}

Do you still get zero for both?

If not, then I think you fell victim to asynchronicity. IIRC, Meteor.subscribe will return immediately and the data/counts will find their way to the browser reactively, but while that's happening your example immediate prints the counts before they've been reactively updated. Since console.log isn't reactive, you only ever see the very first counts of zero. The example above uses a callback to wait until the subscription is ready before fetching the counts.

If I'm right on this, you might try another example that is reactive.

if (Meteor.isClient) {
    Meteor.subscribe('jobCompleted');
    Meteor.subscribe('atDNA');

    Tracker.autorun(function () {
        console.log('Number of jobs completed (client): ', Counts.get('numJobCompleted'));
    });

    Tracker.autorun(function () {
        console.log('Number of atDNA matches (client): ', Counts.get('numAtDNA'));
    });
}
a4xrbj1 commented 9 years ago

Thanks @boxofrox for your effort to help. Your example didn't work either but I made an interesting observation (which I don't understand but you and others might do).

I added a record to Gedmatches collection via the browser console. Then I checked the total number of records of Gedmatches and it still showed 0 in the browser console. Then I checked the same in the MongoDb shell and voila, the newly inserted record is there, increasing the total number by one and I was able to find it as well.

So it's not a reactivity problem, it rather seems that the collections aren't shared at all with the Mini-mongo (again, for reasons that I don't understand).

Hopefully this helps you and you can point me in the right direction.

boxofrox commented 9 years ago

You do seem to have the wierdest problems with Meteor. I've no idea what's going on here, so let's begin.

When you ran your browser record test, did you...

  1. have the insecure Meteor package installed, or was that already removed?
  2. create a collection for Gedmatches on both the client and server? var Gedmatches = new Mongo.Collection('gedmatches');
  3. publish Gedmatches on the server? If yes...
    • what query did you publish?
    • what name did you publish as?
    • are there other published collections with the same name?
  4. subscribe to the name in question 3 in the browser?
  5. use one query/cursor object for both Meteor.publish and Count.publish? You must create separate objects when using both.

Can you generate a small meteor app that reproduces this behavior and publish it in a gist/repo, so I can reproduce the problem myself?

I'll see if I can generate a meteor app based on your first example, and see what happens.

a4xrbj1 commented 9 years ago

Yes, I'm producing weird problems in Meteor. Now for your questions:

A1) The insecure package is still installed A2) It's created in the collections folder (which should create it in both client & server) with Gedmatches = new Meteor.Collection('gedmatches'); A3) Published with Meteor.publish('allGedmatch', function() { Counts.publish(this, 'numGedmatch', Gedmatches.find()); }); allGedmatch and no, no other pub with that same name (checked by finder) A4) Yes, with Meteor.subscribe('allGedmatch', function () { console.log('Number of Gedmatches (client): ', Counts.get('numGedmatch')); }); A5) I don't get your question. But see my code above. I'm not creating an object (to my knowledge)

As far as reproducing the problem in a small app that will take some time and thinking. It's not part of a huge backend app and I just started to work on the frontend part. FYI I'm using Meteor version 1.1.0.3 (if that's important, have to as WebStorm still has problems with Meteor version 1.2)

Your methodological approach to solve the problem is appreciated a lot, if only everyone who created a package would even show half of that effort (I'm getting the runaround by a company dev in another package issue)!

boxofrox commented 9 years ago

All appears to be in order then. These examples should still work with Meteor 1.1.0.3.

I posted an example gist. It mimics the myJobs collection rather than the Gedmatches collection. The differences are trivial, so either should exhibit the same behavior.

Here's how to run it:

  1. clone the gist to a temporary location. e.g. cd /tmp && git clone <gist https clone url> test && cd test.
  2. run the setup.sh to create the app. gists don't support folders, so I had to resort to a script. bash ./setup.sh.
  3. start the app. cd main && meteor run.
  4. open the app url in your browser.

The app has 3 identical collections (ThingsA, ThingsB, ThingsC). They differ only in the method used to populate them which is described in the notes column of the app webpage.

Stop & start the meteor server to change ThingsA. Use the in page buttons to adjust ThingsB or ThingsC.

Each time a record is inserted, a random status of 'completed' or 'in progress' is chosen.

The app displays the counts of all in progress records, all completed records, and the combined total. As records are inserted, the counts adjust accordingly.

Running ThingsA.fetch({}).count() from the browser console returns 0, because I don't publish the collections, therefore the collections aren't cached in the browser, and the records are not accessible. Only the publish-counts counters are sent to the browser by the subscriptions.

While the records are not accessible, I use collection.allow() to allow insertions into the server's collection through the browser's minimongo interface.

I've tested this on 64-bit Linux with the Chromium web browser.

Give that a try and let me know if it works for you. If it doesn't, then I suspect something is wrong with your dev environment.

a4xrbj1 commented 9 years ago

It works like a charm, wow! Actually your approach of only keeping the collection ThingsA on the client (via not publishing) is much better as it save a lot of memory in the browser (my collections will get very big once it's running with more users).

BTW, ThingsA.fetch({}).count() throws an error in the browser console: Uncaught TypeError: ThingsA.fetch is not a function(…) but ThingsA.find({}).count() works (and displays 0 correctly).

I will take a close look at your code and see how I can re-write mine. One thing I noticed already is that you put the pubs under the Meteor.startup function. That's different from my code, not sure if that has any implications (it means the pub is executed last in your example, right?)

What I don't yet understand is how you get from Counts.publish(this, 'things-a-completed-count', ThingsA.find({ status: { $eq: 'completed' } })); where you define the variable things-a-completed-count and it's been given to the template <td>{{ counter (inProgressCounterName) }}</td>

You've lost me here, guess some debugging is in order. Many thanks for setting this up and also your help on the other mdbutton issue!

boxofrox commented 9 years ago

Oops. I meant ThingsA.find({}).count().

Meteor.startup does run the callback last on the server. According to a forum comment [1], you may run into problems with Meteor's goofy file load order running the main app files before the collection files, so Meteor.startup helps avoid this situation. In my example it's pretty useless because main.js loads last in the file load order. You can put console.log('loading file xxx') at the top of your source files to see which order they are loaded in. If collections are created after the source files that use them, then Meteor.startup will defer the source execution until after the collection vars are initialized.

What I don't yet understand is how you get from Counts.publish(this, 'things-a-completed-count', ThingsA.find({ status: { $eq: 'completed' } })); where you define the variable things-a-completed-count and it's been given to the template <td>{{ counter (inProgressCounterName) }}</td>

That part's a bit convoluted and what I think of as spacebars magic. Spacebars appears to have very narrow scope (what code is visible to templates) and its docs speak of a data context which is similar to the this automatic variable we have in our function calls. Here my best attempt at explaining how it works here...