Problem with the Evidence class

[jdramsey]

Alright, so I've spent a bunch of time wading through the existing code and trying to make sense of things. I've come to the conclusion that either (A) for some reason that's not mentioned in the preamble comment, rowsummingexactupdater is not actually intended for my use case, or (B) the Evidence class is broken. Basically, when I run that updater using an evidence object constructed from my proposition that [these nodes] can only take on [these categories], it returns a null pointer error complaining that the manipulation part of the evidence is null. However, the only way I can find in the evidence class to generate a non-null manipulation forces the evidence's proposition to be a tautology. And I can find no way to either (1) update an evidence object's proposition after it has been created, or (2) populate an evidence object's null manipulation with a non-null manipulation (manipulation values can be modified, but not created fresh).

So! That's pretty much the short of it. What I want to do is modify the evidence class to include, at a minimum, a constructor that will accept both a manipulation and a proposition. From what I can tell, an evidence is basically just the a proposition + manipulation pairing anyway, so I don't know why this isn't already the case. I'm probably just completely not understanding something about Java, and there's an easy solution to this mess that doesn't involve me probably screwing up your existing code, so I wanted to get your thoughts on all this first. I've spent a bit of time searching the code for other places where the evidence class is being used so I can see how you might've gotten around this somewhere else, but was unable to find anything.

[ekummerfeld]

This is from an e-mail I wrote. I haven't pushed my code onto my own branch yet, will do that later today, if it helps.

[ekummerfeld]

I just updated my own branch at https://github.com/ekummerfeld/tetrad

You can find the resimulate code that isn't quite working here: https://github.com/ekummerfeld/tetrad/blob/master/tetrad-lib/src/main/java/edu/cmu/tetrad/simulation/Resimulate.java

The test file I use is here: https://github.com/ekummerfeld/tetrad/blob/master/tetrad-lib/src/test/java/edu/cmu/tetrad/test/testResimulate.java

If you run the test file, you will see the null pointer exception thrown when the updater tries to update using the evidence object I made. It complains that the manipulation is null. I can find no way to make an evidence object that both has the proposition that I want, and a non-null manipulation. I'm not actually manipulating any variables, so it would just be an array of falses. The evidence object is made at line 230, and the updater tries to use it, but fails, at line 238.

[jdramsey]

Hmm.. I do believe it should work, since it is working in the interface for the updater. I wonder.

On Tue, Dec 15, 2015 at 12:32 PM, ekummerfeld notifications@github.com wrote:

I just updated my own branch at https://github.com/ekummerfeld/tetrad

You can find the resimulate code that isn't quite working here: https://github.com/ekummerfeld/tetrad/blob/master/tetrad-lib/src/main/java/edu/cmu/tetrad/simulation/Resimulate.java

The test file I use is here: https://github.com/ekummerfeld/tetrad/blob/master/tetrad-lib/src/test/java/edu/cmu/tetrad/test/testResimulate.java

If you run the test file, you will see the null pointer exception thrown when the updater tries to update using the evidence object I made. It complains that the manipulation is null. I can find no way to make an evidence object that both has the proposition that I want, and a non-null manipulation. I'm not actually manipulating any variables, so it would just be an array of falses. The evidence object is made at line 230, and the updater tries to use it, but fails, at line 238.

— Reply to this email directly or view it on GitHub https://github.com/cmu-phil/tetrad/issues/106#issuecomment-164833581.

Joseph D. Ramsey Special Faculty and Director of Research Computing Department of Philosophy 135 Baker Hall Carnegie Mellon University Pittsburgh, PA 15213

jsph.ramsey@gmail.com Office: (412) 268-8063 http://www.andrew.cmu.edu/user/jdramsey

[jdramsey]

Try now. I changed a few things. I may have screwed up the actual calculation you were doing, since I didn't understand it.

On Tue, Dec 15, 2015 at 2:57 PM, Joseph Ramsey jdramsey@andrew.cmu.edu wrote:

Hmm.. I do believe it should work, since it is working in the interface for the updater. I wonder.

On Tue, Dec 15, 2015 at 12:32 PM, ekummerfeld notifications@github.com wrote:

I just updated my own branch at https://github.com/ekummerfeld/tetrad

You can find the resimulate code that isn't quite working here: https://github.com/ekummerfeld/tetrad/blob/master/tetrad-lib/src/main/java/edu/cmu/tetrad/simulation/Resimulate.java

The test file I use is here: https://github.com/ekummerfeld/tetrad/blob/master/tetrad-lib/src/test/java/edu/cmu/tetrad/test/testResimulate.java

If you run the test file, you will see the null pointer exception thrown when the updater tries to update using the evidence object I made. It complains that the manipulation is null. I can find no way to make an evidence object that both has the proposition that I want, and a non-null manipulation. I'm not actually manipulating any variables, so it would just be an array of falses. The evidence object is made at line 230, and the updater tries to use it, but fails, at line 238.

— Reply to this email directly or view it on GitHub https://github.com/cmu-phil/tetrad/issues/106#issuecomment-164833581.

Joseph D. Ramsey Special Faculty and Director of Research Computing Department of Philosophy 135 Baker Hall Carnegie Mellon University Pittsburgh, PA 15213

jsph.ramsey@gmail.com Office: (412) 268-8063 http://www.andrew.cmu.edu/user/jdramsey

Joseph D. Ramsey Special Faculty and Director of Research Computing Department of Philosophy 135 Baker Hall Carnegie Mellon University Pittsburgh, PA 15213

jsph.ramsey@gmail.com Office: (412) 268-8063 http://www.andrew.cmu.edu/user/jdramsey

[ekummerfeld]

I'm not sure what I should be trying. Should I pull from the master branch? Did you do something to the Evidence class?

[ekummerfeld]

I'm gonna try that and see what happens.

[ekummerfeld]

git pull on the cmu-phil branch doesn't make any changes for me

[ekummerfeld]

I made a change to the testResimulate file since it looks like changes were made to randomgraphforwardedges since I last pulled from the master branch, just copied the new method call from the current TestFgs file. Now that testResimulate can generate data again, I reran it, and still get the same null pointer exception. Rowsummingexactupdater tries to set its own evidence -> tries to create a manipulated graph -> checks the evidence for manipulation via the isManipulated method, finds a null value. Either Rowsummingexactupdater needs to relax about null manipulations, or I need to be able to make an Evidence object with a specified proposition AND manipulation.

[jdramsey]

Oh maybe I need to push, just a sec...

On Tue, Dec 15, 2015 at 4:33 PM, ekummerfeld notifications@github.com wrote:

I made a change to the testResimulate file since it looks like changes were made to randomgraphforwardedges since I last pulled from the master branch, just copied the new method call from the current TestFgs file. Now that testResimulate can generate data again, I reran it, and still get the same null pointer exception. Rowsummingexactupdater tries to set its own evidence -> tries to create a manipulated graph -> checks the evidence for manipulation via the isManipulated method, finds a null value. Either Rowsummingexactupdater needs to relax about null manipulations, or I need to be able to make an Evidence object with a specified proposition AND manipulation.

— Reply to this email directly or view it on GitHub https://github.com/cmu-phil/tetrad/issues/106#issuecomment-164904146.

Joseph D. Ramsey Special Faculty and Director of Research Computing Department of Philosophy 135 Baker Hall Carnegie Mellon University Pittsburgh, PA 15213

jsph.ramsey@gmail.com Office: (412) 268-8063 http://www.andrew.cmu.edu/user/jdramsey

[jdramsey]

You may need to make me a collaborator on your fork.

J

On Tue, Dec 15, 2015 at 5:07 PM, Joseph Ramsey jdramsey@andrew.cmu.edu wrote:

Oh maybe I need to push, just a sec...

On Tue, Dec 15, 2015 at 4:33 PM, ekummerfeld notifications@github.com wrote:

I made a change to the testResimulate file since it looks like changes were made to randomgraphforwardedges since I last pulled from the master branch, just copied the new method call from the current TestFgs file. Now that testResimulate can generate data again, I reran it, and still get the same null pointer exception. Rowsummingexactupdater tries to set its own evidence -> tries to create a manipulated graph -> checks the evidence for manipulation via the isManipulated method, finds a null value. Either Rowsummingexactupdater needs to relax about null manipulations, or I need to be able to make an Evidence object with a specified proposition AND manipulation.

— Reply to this email directly or view it on GitHub https://github.com/cmu-phil/tetrad/issues/106#issuecomment-164904146.

Joseph D. Ramsey Special Faculty and Director of Research Computing Department of Philosophy 135 Baker Hall Carnegie Mellon University Pittsburgh, PA 15213

jsph.ramsey@gmail.com Office: (412) 268-8063 http://www.andrew.cmu.edu/user/jdramsey

Joseph D. Ramsey Special Faculty and Director of Research Computing Department of Philosophy 135 Baker Hall Carnegie Mellon University Pittsburgh, PA 15213

jsph.ramsey@gmail.com Office: (412) 268-8063 http://www.andrew.cmu.edu/user/jdramsey

[ekummerfeld]

Not sure how that works. Can you fork my branch and make a pull request?

[jdramsey]

From what it looks like in the interface forking would fork the original repository, not the fork. I know there are 18 forks in the original repository and it says there are 18 forks on your page. By law of strange numbers I conclude it is not a coincindence and your fork has not gotten 18 forks. :)

I think if you just go into the settings on your page and add me as a collaborator it should work. Should be self explanatory. Just don't delete the fork. :)

Joe

On Tue, Dec 15, 2015 at 6:08 PM, ekummerfeld notifications@github.com wrote:

Not sure how that works. Can you fork my branch and make a pull request?

— Reply to this email directly or view it on GitHub https://github.com/cmu-phil/tetrad/issues/106#issuecomment-164919277.

Joseph D. Ramsey Special Faculty and Director of Research Computing Department of Philosophy 135 Baker Hall Carnegie Mellon University Pittsburgh, PA 15213

jsph.ramsey@gmail.com Office: (412) 268-8063 http://www.andrew.cmu.edu/user/jdramsey

[jdramsey]

But I may misunderstand.

On Tue, Dec 15, 2015 at 6:21 PM, Joseph Ramsey jdramsey@andrew.cmu.edu wrote:

From what it looks like in the interface forking would fork the original repository, not the fork. I know there are 18 forks in the original repository and it says there are 18 forks on your page. By law of strange numbers I conclude it is not a coincindence and your fork has not gotten 18 forks. :)

I think if you just go into the settings on your page and add me as a collaborator it should work. Should be self explanatory. Just don't delete the fork. :)

Joe

On Tue, Dec 15, 2015 at 6:08 PM, ekummerfeld notifications@github.com wrote:

Not sure how that works. Can you fork my branch and make a pull request?

— Reply to this email directly or view it on GitHub https://github.com/cmu-phil/tetrad/issues/106#issuecomment-164919277.

Joseph D. Ramsey Special Faculty and Director of Research Computing Department of Philosophy 135 Baker Hall Carnegie Mellon University Pittsburgh, PA 15213

jsph.ramsey@gmail.com Office: (412) 268-8063 http://www.andrew.cmu.edu/user/jdramsey

Joseph D. Ramsey Special Faculty and Director of Research Computing Department of Philosophy 135 Baker Hall Carnegie Mellon University Pittsburgh, PA 15213

jsph.ramsey@gmail.com Office: (412) 268-8063 http://www.andrew.cmu.edu/user/jdramsey

[jdramsey]

I guess I could try to figure out which files I changed.

On Tue, Dec 15, 2015 at 6:22 PM, Joseph Ramsey jdramsey@andrew.cmu.edu wrote:

But I may misunderstand.

On Tue, Dec 15, 2015 at 6:21 PM, Joseph Ramsey jdramsey@andrew.cmu.edu wrote:

From what it looks like in the interface forking would fork the original repository, not the fork. I know there are 18 forks in the original repository and it says there are 18 forks on your page. By law of strange numbers I conclude it is not a coincindence and your fork has not gotten 18 forks. :)

I think if you just go into the settings on your page and add me as a collaborator it should work. Should be self explanatory. Just don't delete the fork. :)

Joe

On Tue, Dec 15, 2015 at 6:08 PM, ekummerfeld notifications@github.com wrote:

Not sure how that works. Can you fork my branch and make a pull request?

— Reply to this email directly or view it on GitHub https://github.com/cmu-phil/tetrad/issues/106#issuecomment-164919277.

Joseph D. Ramsey Special Faculty and Director of Research Computing Department of Philosophy 135 Baker Hall Carnegie Mellon University Pittsburgh, PA 15213

jsph.ramsey@gmail.com Office: (412) 268-8063 http://www.andrew.cmu.edu/user/jdramsey

Joseph D. Ramsey Special Faculty and Director of Research Computing Department of Philosophy 135 Baker Hall Carnegie Mellon University Pittsburgh, PA 15213

jsph.ramsey@gmail.com Office: (412) 268-8063 http://www.andrew.cmu.edu/user/jdramsey

Joseph D. Ramsey Special Faculty and Director of Research Computing Department of Philosophy 135 Baker Hall Carnegie Mellon University Pittsburgh, PA 15213

jsph.ramsey@gmail.com Office: (412) 268-8063 http://www.andrew.cmu.edu/user/jdramsey

[ekummerfeld]

Okay I added you as a collaborator.

[ekummerfeld]

In theory there are only 2 files to change: Resimulation and testResimulation

[jdramsey]

Pushed!

On Tue, Dec 15, 2015 at 7:18 PM, ekummerfeld notifications@github.com wrote:

In theory there are only 2 files to change: Resimulation and testResimulation

— Reply to this email directly or view it on GitHub https://github.com/cmu-phil/tetrad/issues/106#issuecomment-164941163.

Joseph D. Ramsey Special Faculty and Director of Research Computing Department of Philosophy 135 Baker Hall Carnegie Mellon University Pittsburgh, PA 15213

jsph.ramsey@gmail.com Office: (412) 268-8063 http://www.andrew.cmu.edu/user/jdramsey

[jdramsey]

OK now you can make me a non-collaborator if you want. :)

On Tue, Dec 15, 2015 at 7:20 PM, Joseph Ramsey jdramsey@andrew.cmu.edu wrote:

Pushed!

On Tue, Dec 15, 2015 at 7:18 PM, ekummerfeld notifications@github.com wrote:

In theory there are only 2 files to change: Resimulation and testResimulation

— Reply to this email directly or view it on GitHub https://github.com/cmu-phil/tetrad/issues/106#issuecomment-164941163.

Joseph D. Ramsey Special Faculty and Director of Research Computing Department of Philosophy 135 Baker Hall Carnegie Mellon University Pittsburgh, PA 15213

jsph.ramsey@gmail.com Office: (412) 268-8063 http://www.andrew.cmu.edu/user/jdramsey

Joseph D. Ramsey Special Faculty and Director of Research Computing Department of Philosophy 135 Baker Hall Carnegie Mellon University Pittsburgh, PA 15213

jsph.ramsey@gmail.com Office: (412) 268-8063 http://www.andrew.cmu.edu/user/jdramsey

[ekummerfeld]

done!

I'll pull the changes and see what's what.

[ekummerfeld]

I don't understand what happened to the testresimulate file, but my intelliJ is really unhappy with the class using an upper case T while the file uses a lower case t.

[ekummerfeld]

So it looks like there was a way for me to fiddle with the proposition inside the evidence after all?

[ekummerfeld]

so I renamed that file in order to run it, but it's still not working. Now getMarginal is returning NaN, which is naturally causing problems. Did this code work for you?

[ekummerfeld]

I added a few println and you can see the NaN values pop up instead of probability doubles when getMarginal tries to do its stuff.

[ekummerfeld]

meant to mention: I just pushed that back onto github

[jdramsey]

Yeah, the evidence has the manipulation in it.

On Tue, Dec 15, 2015 at 9:35 PM, ekummerfeld notifications@github.com wrote:

So it looks like there was a way for me to fiddle with the proposition inside the evidence after all?

— Reply to this email directly or view it on GitHub https://github.com/cmu-phil/tetrad/issues/106#issuecomment-164967557.

Joseph D. Ramsey Special Faculty and Director of Research Computing Department of Philosophy 135 Baker Hall Carnegie Mellon University Pittsburgh, PA 15213

jsph.ramsey@gmail.com Office: (412) 268-8063 http://www.andrew.cmu.edu/user/jdramsey

[jdramsey]

I fixed a couple of -99 missing value problems, possibly in the wrong way, but it runs for me at least. I can't push anymore so I'm attaching the file.

Joe

On Wed, Dec 16, 2015 at 6:01 AM, Joseph Ramsey jdramsey@andrew.cmu.edu wrote:

Yeah, the evidence has the manipulation in it.

On Tue, Dec 15, 2015 at 9:35 PM, ekummerfeld notifications@github.com wrote:

So it looks like there was a way for me to fiddle with the proposition inside the evidence after all?

— Reply to this email directly or view it on GitHub https://github.com/cmu-phil/tetrad/issues/106#issuecomment-164967557.

Joseph D. Ramsey Special Faculty and Director of Research Computing Department of Philosophy 135 Baker Hall Carnegie Mellon University Pittsburgh, PA 15213

jsph.ramsey@gmail.com Office: (412) 268-8063 http://www.andrew.cmu.edu/user/jdramsey

Joseph D. Ramsey Special Faculty and Director of Research Computing Department of Philosophy 135 Baker Hall Carnegie Mellon University Pittsburgh, PA 15213

jsph.ramsey@gmail.com Office: (412) 268-8063 http://www.andrew.cmu.edu/user/jdramsey

[jdramsey]

By the way I tried forking your repository but it just pointed me to the fork I already had in my account of Tetrad. :( I guess that's GitHub's political statement.

Joe

On Wed, Dec 16, 2015 at 6:15 AM, Joseph Ramsey jdramsey@andrew.cmu.edu wrote:

I fixed a couple of -99 missing value problems, possibly in the wrong way, but it runs for me at least. I can't push anymore so I'm attaching the file.

Joe

On Wed, Dec 16, 2015 at 6:01 AM, Joseph Ramsey jdramsey@andrew.cmu.edu wrote:

Yeah, the evidence has the manipulation in it.

On Tue, Dec 15, 2015 at 9:35 PM, ekummerfeld notifications@github.com wrote:

So it looks like there was a way for me to fiddle with the proposition inside the evidence after all?

— Reply to this email directly or view it on GitHub https://github.com/cmu-phil/tetrad/issues/106#issuecomment-164967557.

Joseph D. Ramsey Special Faculty and Director of Research Computing Department of Philosophy 135 Baker Hall Carnegie Mellon University Pittsburgh, PA 15213

jsph.ramsey@gmail.com Office: (412) 268-8063 http://www.andrew.cmu.edu/user/jdramsey

Joseph D. Ramsey Special Faculty and Director of Research Computing Department of Philosophy 135 Baker Hall Carnegie Mellon University Pittsburgh, PA 15213

jsph.ramsey@gmail.com Office: (412) 268-8063 http://www.andrew.cmu.edu/user/jdramsey

Joseph D. Ramsey Special Faculty and Director of Research Computing Department of Philosophy 135 Baker Hall Carnegie Mellon University Pittsburgh, PA 15213

jsph.ramsey@gmail.com Office: (412) 268-8063 http://www.andrew.cmu.edu/user/jdramsey

[ekummerfeld]

I don't see a file attached here or in e-mail, not sure where else to look. I'll just add you as a collaborator and leave it at that for a while, seems like the easiest way.

[jdramsey]

OK, pushed. I did attach it somewhere, maybe to a ghost.

On Wed, Dec 16, 2015 at 7:28 AM, ekummerfeld notifications@github.com wrote:

I don't see a file attached here or in e-mail, not sure where else to look. I'll just add you as a collaborator and leave it at that for a while, seems like the easiest way.

— Reply to this email directly or view it on GitHub https://github.com/cmu-phil/tetrad/issues/106#issuecomment-165091762.

Joseph D. Ramsey Special Faculty and Director of Research Computing Department of Philosophy 135 Baker Hall Carnegie Mellon University Pittsburgh, PA 15213

jsph.ramsey@gmail.com Office: (412) 268-8063 http://www.andrew.cmu.edu/user/jdramsey

[ekummerfeld]

throwing in the continues lets the code complete but doesn't fix the problem where

double probability = conditionUpdate.getMarginal(nodeXindex, i);

is returning NaN rather than a number.

[jdramsey]

Sorry I can't help at the moment, up to my elbows in another problem but I'll get to it later. I can't get it to happen myself though.

On Wed, Dec 16, 2015 at 9:03 AM, ekummerfeld notifications@github.com wrote:

throwing in the continues lets the code complete but doesn't fix the problem where

double probability = conditionUpdate.getMarginal(nodeXindex, i);

is returning NaN rather than a number.

— Reply to this email directly or view it on GitHub https://github.com/cmu-phil/tetrad/issues/106#issuecomment-165114191.

Joseph D. Ramsey Special Faculty and Director of Research Computing Department of Philosophy 135 Baker Hall Carnegie Mellon University Pittsburgh, PA 15213

jsph.ramsey@gmail.com Office: (412) 268-8063 http://www.andrew.cmu.edu/user/jdramsey

[ekummerfeld]

No problem. Let me know when's a good time for you. Might be worth talking over voip or something.

[jdramsey]

Do you mean this?

        int newXvalue = -99;
        //now iterate through the categories to see which one owns that probability mass real estate
        for (int i=0; i < numCat; i++ ){
            //for each category, calc the marginal conditional prob of nodeX having that value
            double probability = conditionUpdate.getMarginal(nodeXindex, i);
            System.out.println("cat " + i + " prob " + probability);
            sum += probability;

            if (sum >= cutoff) {
                newXvalue = i;
                break;
            }
        }

newXValue there the first time through is -99, undefined, so the marginal is undefined...?

[ekummerfeld]

The marginal output by the getMarginal method is undefined, yes, so newXvalue never gets updated with a new value.

[jdramsey]

Sorry, department party (which you didn't come to!--just kidding, no guilt . :)) Could you update newXvalue? Or did you already?

[ekummerfeld]

double probability = conditionUpdate.getMarginal(nodeXindex, i); System.out.println("cat " + i + " prob " + probability); sum += probability;

        if (sum >= cutoff) {
            newXvalue = i;
            break;
        }

probability gets the value NaN, so sum never gets big, so sum is never greater than cutoff, so newXvalue = i never occurs because the if clause is never true. The problem is that conditionUpdate.getMarginal(nodeXindex, i); is returning NaN values. I have no idea why.

[jdramsey]

Hmm.. this is what I see:

int newXvalue = -99; //now iterate through the categories to see which one owns that probability mass real estate for (int i=0; i < numCat; i++ ){ //for each category, calc the marginal conditional prob of nodeX having that value double probability = conditionUpdate.getMarginal(nodeXindex, i); System.out.println("cat " + i + " prob " + probability); sum += probability;

if (sum >= cutoff) {
    newXvalue = i;
    break;
}

}

newXvalue is set to -99, and then right inside the loop you call

conditionUpdate.getMarginal(nodeXindex, i);

i.e.

conditionUpdate.getMarginal(-99, i);

This is undefined and so returns NaN.

On Wed, Dec 16, 2015 at 3:00 PM, ekummerfeld notifications@github.com wrote:

double probability = conditionUpdate.getMarginal(nodeXindex, i); System.out.println("cat " + i + " prob " + probability); sum += probability;

    if (sum >= cutoff) {
        newXvalue = i;
        break;
    }

probability gets the value NaN, so sum never gets big, so sum is never greater than cutoff, so newXvalue = i never occurs because the if clause is never true. The problem is that conditionUpdate.getMarginal(nodeXindex, i); is returning NaN values. I have no idea why.

— Reply to this email directly or view it on GitHub https://github.com/cmu-phil/tetrad/issues/106#issuecomment-165226259.

Joseph D. Ramsey Special Faculty and Director of Research Computing Department of Philosophy 135 Baker Hall Carnegie Mellon University Pittsburgh, PA 15213

jsph.ramsey@gmail.com Office: (412) 268-8063 http://www.andrew.cmu.edu/user/jdramsey

[jdramsey]

I'm not smart enough to know what it should say, sorry.

On Wed, Dec 16, 2015 at 3:06 PM, Joseph Ramsey jdramsey@andrew.cmu.edu wrote:

Hmm.. this is what I see:

int newXvalue = -99; //now iterate through the categories to see which one owns that probability mass real estate for (int i=0; i < numCat; i++ ){ //for each category, calc the marginal conditional prob of nodeX having that value double probability = conditionUpdate.getMarginal(nodeXindex, i); System.out.println("cat " + i + " prob " + probability); sum += probability;

if (sum >= cutoff) {
    newXvalue = i;
    break;
}

}

newXvalue is set to -99, and then right inside the loop you call

conditionUpdate.getMarginal(nodeXindex, i);

i.e.

conditionUpdate.getMarginal(-99, i);

This is undefined and so returns NaN.

On Wed, Dec 16, 2015 at 3:00 PM, ekummerfeld notifications@github.com wrote:

double probability = conditionUpdate.getMarginal(nodeXindex, i); System.out.println("cat " + i + " prob " + probability); sum += probability;

    if (sum >= cutoff) {
        newXvalue = i;
        break;
    }

probability gets the value NaN, so sum never gets big, so sum is never greater than cutoff, so newXvalue = i never occurs because the if clause is never true. The problem is that conditionUpdate.getMarginal(nodeXindex, i); is returning NaN values. I have no idea why.

— Reply to this email directly or view it on GitHub https://github.com/cmu-phil/tetrad/issues/106#issuecomment-165226259.

Joseph D. Ramsey Special Faculty and Director of Research Computing Department of Philosophy 135 Baker Hall Carnegie Mellon University Pittsburgh, PA 15213

jsph.ramsey@gmail.com Office: (412) 268-8063 http://www.andrew.cmu.edu/user/jdramsey

Joseph D. Ramsey Special Faculty and Director of Research Computing Department of Philosophy 135 Baker Hall Carnegie Mellon University Pittsburgh, PA 15213

jsph.ramsey@gmail.com Office: (412) 268-8063 http://www.andrew.cmu.edu/user/jdramsey

[jdramsey]

Hold on, that's not right... just a sec..

On Wed, Dec 16, 2015 at 3:06 PM, Joseph Ramsey jdramsey@andrew.cmu.edu wrote:

I'm not smart enough to know what it should say, sorry.

On Wed, Dec 16, 2015 at 3:06 PM, Joseph Ramsey jdramsey@andrew.cmu.edu wrote:

Hmm.. this is what I see:

int newXvalue = -99; //now iterate through the categories to see which one owns that probability mass real estate for (int i=0; i < numCat; i++ ){ //for each category, calc the marginal conditional prob of nodeX having that value double probability = conditionUpdate.getMarginal(nodeXindex, i); System.out.println("cat " + i + " prob " + probability); sum += probability;

if (sum >= cutoff) {
    newXvalue = i;
    break;
}

}

newXvalue is set to -99, and then right inside the loop you call

conditionUpdate.getMarginal(nodeXindex, i);

i.e.

conditionUpdate.getMarginal(-99, i);

This is undefined and so returns NaN.

On Wed, Dec 16, 2015 at 3:00 PM, ekummerfeld notifications@github.com wrote:

double probability = conditionUpdate.getMarginal(nodeXindex, i); System.out.println("cat " + i + " prob " + probability); sum += probability;

    if (sum >= cutoff) {
        newXvalue = i;
        break;
    }

probability gets the value NaN, so sum never gets big, so sum is never greater than cutoff, so newXvalue = i never occurs because the if clause is never true. The problem is that conditionUpdate.getMarginal(nodeXindex, i); is returning NaN values. I have no idea why.

— Reply to this email directly or view it on GitHub https://github.com/cmu-phil/tetrad/issues/106#issuecomment-165226259.

Joseph D. Ramsey Special Faculty and Director of Research Computing Department of Philosophy 135 Baker Hall Carnegie Mellon University Pittsburgh, PA 15213

jsph.ramsey@gmail.com Office: (412) 268-8063 http://www.andrew.cmu.edu/user/jdramsey

Joseph D. Ramsey Special Faculty and Director of Research Computing Department of Philosophy 135 Baker Hall Carnegie Mellon University Pittsburgh, PA 15213

jsph.ramsey@gmail.com Office: (412) 268-8063 http://www.andrew.cmu.edu/user/jdramsey

Joseph D. Ramsey Special Faculty and Director of Research Computing Department of Philosophy 135 Baker Hall Carnegie Mellon University Pittsburgh, PA 15213

jsph.ramsey@gmail.com Office: (412) 268-8063 http://www.andrew.cmu.edu/user/jdramsey

[jdramsey]

Well there's this:

double cellProb = getCellProb(variableValues);

in BayesImProbs is returning NaN. Do you have a Bayes IM that's not completely filled in?

On Wed, Dec 16, 2015 at 3:07 PM, Joseph Ramsey jdramsey@andrew.cmu.edu wrote:

Hold on, that's not right... just a sec..

On Wed, Dec 16, 2015 at 3:06 PM, Joseph Ramsey jdramsey@andrew.cmu.edu wrote:

I'm not smart enough to know what it should say, sorry.

On Wed, Dec 16, 2015 at 3:06 PM, Joseph Ramsey jdramsey@andrew.cmu.edu wrote:

Hmm.. this is what I see:

int newXvalue = -99; //now iterate through the categories to see which one owns that probability mass real estate for (int i=0; i < numCat; i++ ){ //for each category, calc the marginal conditional prob of nodeX having that value double probability = conditionUpdate.getMarginal(nodeXindex, i); System.out.println("cat " + i + " prob " + probability); sum += probability;

if (sum >= cutoff) {
    newXvalue = i;
    break;
}

}

newXvalue is set to -99, and then right inside the loop you call

conditionUpdate.getMarginal(nodeXindex, i);

i.e.

conditionUpdate.getMarginal(-99, i);

This is undefined and so returns NaN.

On Wed, Dec 16, 2015 at 3:00 PM, ekummerfeld notifications@github.com wrote:

double probability = conditionUpdate.getMarginal(nodeXindex, i); System.out.println("cat " + i + " prob " + probability); sum += probability;

    if (sum >= cutoff) {
        newXvalue = i;
        break;
    }

probability gets the value NaN, so sum never gets big, so sum is never greater than cutoff, so newXvalue = i never occurs because the if clause is never true. The problem is that conditionUpdate.getMarginal(nodeXindex, i); is returning NaN values. I have no idea why.

— Reply to this email directly or view it on GitHub https://github.com/cmu-phil/tetrad/issues/106#issuecomment-165226259.

Joseph D. Ramsey Special Faculty and Director of Research Computing Department of Philosophy 135 Baker Hall Carnegie Mellon University Pittsburgh, PA 15213

jsph.ramsey@gmail.com Office: (412) 268-8063 http://www.andrew.cmu.edu/user/jdramsey

Joseph D. Ramsey Special Faculty and Director of Research Computing Department of Philosophy 135 Baker Hall Carnegie Mellon University Pittsburgh, PA 15213

jsph.ramsey@gmail.com Office: (412) 268-8063 http://www.andrew.cmu.edu/user/jdramsey

Joseph D. Ramsey Special Faculty and Director of Research Computing Department of Philosophy 135 Baker Hall Carnegie Mellon University Pittsburgh, PA 15213

jsph.ramsey@gmail.com Office: (412) 268-8063 http://www.andrew.cmu.edu/user/jdramsey

Joseph D. Ramsey Special Faculty and Director of Research Computing Department of Philosophy 135 Baker Hall Carnegie Mellon University Pittsburgh, PA 15213

jsph.ramsey@gmail.com Office: (412) 268-8063 http://www.andrew.cmu.edu/user/jdramsey

[ekummerfeld]

I don't know. It SHOULD be completely filled. A few lines up it gets fit to data, and then updated.

[jdramsey]

Is there enough data to fill in all of the cells?

On Wed, Dec 16, 2015 at 4:01 PM, ekummerfeld notifications@github.com wrote:

I don't know. It SHOULD be completely filled. A few lines up it gets fit to data, and then updated.

— Reply to this email directly or view it on GitHub https://github.com/cmu-phil/tetrad/issues/106#issuecomment-165238262.

Joseph D. Ramsey Special Faculty and Director of Research Computing Department of Philosophy 135 Baker Hall Carnegie Mellon University Pittsburgh, PA 15213

jsph.ramsey@gmail.com Office: (412) 268-8063 http://www.andrew.cmu.edu/user/jdramsey

[ekummerfeld]

probably not without a dirichlet prior, which is why I used the dirichlet bayes im to fit the data first. possibly I'm doing that wrong?

[jdramsey]

Try it this way (just pushed). It looked like you were making a blank Dirichlet bayes im (all zeros) instead of a symmetric one with value 1 (all 1's).

J

On Wed, Dec 16, 2015 at 4:50 PM, ekummerfeld notifications@github.com wrote:

probably not without a dirichlet prior, which is why I used the dirichlet bayes im to fit the data first. possibly I'm doing that wrong?

— Reply to this email directly or view it on GitHub https://github.com/cmu-phil/tetrad/issues/106#issuecomment-165253810.

Joseph D. Ramsey Special Faculty and Director of Research Computing Department of Philosophy 135 Baker Hall Carnegie Mellon University Pittsburgh, PA 15213

jsph.ramsey@gmail.com Office: (412) 268-8063 http://www.andrew.cmu.edu/user/jdramsey

[ekummerfeld]

Ah, that would be a problem wouldn't it! Thanks for spotting that, I'll pull it and get back to you.

[ekummerfeld]

IT'S ALIIIIIIVE!!!!

[jdramsey]

Yay!

On Wed, Dec 16, 2015 at 8:39 PM, ekummerfeld notifications@github.com wrote:

IT'S ALIIIIIIVE!!!!

— Reply to this email directly or view it on GitHub https://github.com/cmu-phil/tetrad/issues/106#issuecomment-165310253.

Joseph D. Ramsey Special Faculty and Director of Research Computing Department of Philosophy 135 Baker Hall Carnegie Mellon University Pittsburgh, PA 15213

jsph.ramsey@gmail.com Office: (412) 268-8063 http://www.andrew.cmu.edu/user/jdramsey