Rename: Display informative exception when singularity exceptions are encountered in all public searches.

[cg09]

At one time we computed correlations "on the fly" as they were needed for tests. Wouldn't that make looking for non-singular subsets unnecessary?

On Fri, Apr 14, 2023 at 12:00 PM Joseph Ramsey @.***> wrote:

— Reply to this email directly, view it on GitHub https://github.com/cmu-phil/tetrad/issues/1589, or unsubscribe https://github.com/notifications/unsubscribe-auth/AD4Y3OJUXGWUEQWYPKLWO63XBFYAHANCNFSM6AAAAAAW6TF6GA . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[jdramsey]

No, that was just another method for calculating the correlations for a correlation matrix. The whole correlation matrix could still be singular once all of the correlations are calculated.

[cg09]

Of course it could, but why would you care??

On Fri, Apr 14, 2023 at 4:49 PM Joseph Ramsey @.***> wrote:

No, that was just another method for calculating the correlations for a correlation matrix. The whole correlation matrix could still be singular once all of the correlations are calculated.

— Reply to this email directly, view it on GitHub https://github.com/cmu-phil/tetrad/issues/1589#issuecomment-1509243897, or unsubscribe https://github.com/notifications/unsubscribe-auth/AD4Y3OMXM5JCR2IP3MM6JC3XBGZ5XANCNFSM6AAAAAAW6TF6GA . You are receiving this because you commented.Message ID: @.***>

[jdramsey]

Oh--that leads to singularity exceptions--matrices or submatrices that can't be inverted. Then you can't do estimations or sometimes can't even evaluate tests or scores.

On Fri, Apr 14, 2023 at 5:01 PM cg09 @.***> wrote:

Of course it could, but why would you care??

On Fri, Apr 14, 2023 at 4:49 PM Joseph Ramsey @.***> wrote:

No, that was just another method for calculating the correlations for a correlation matrix. The whole correlation matrix could still be singular once all of the correlations are calculated.

— Reply to this email directly, view it on GitHub <https://github.com/cmu-phil/tetrad/issues/1589#issuecomment-1509243897 , or unsubscribe < https://github.com/notifications/unsubscribe-auth/AD4Y3OMXM5JCR2IP3MM6JC3XBGZ5XANCNFSM6AAAAAAW6TF6GA

. You are receiving this because you commented.Message ID: @.***>

— Reply to this email directly, view it on GitHub https://github.com/cmu-phil/tetrad/issues/1589#issuecomment-1509260668, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACLFSR76OF2GZMRYBGALBJLXBG3MHANCNFSM6AAAAAAW6TF6GA . You are receiving this because you authored the thread.Message ID: @.***>

[cg09]

yes, I forget about parameter estimation.

On Fri, Apr 14, 2023 at 5:48 PM Joseph Ramsey @.***> wrote:

Oh--that leads to singularity exceptions--matrices or submatrices that can't be inverted. Then you can't do estimations or sometimes can't even evaluate tests or scores.

On Fri, Apr 14, 2023 at 5:01 PM cg09 @.***> wrote:

Of course it could, but why would you care??

On Fri, Apr 14, 2023 at 4:49 PM Joseph Ramsey @.***> wrote:

No, that was just another method for calculating the correlations for a correlation matrix. The whole correlation matrix could still be singular once all of the correlations are calculated.

— Reply to this email directly, view it on GitHub < https://github.com/cmu-phil/tetrad/issues/1589#issuecomment-1509243897 , or unsubscribe <

https://github.com/notifications/unsubscribe-auth/AD4Y3OMXM5JCR2IP3MM6JC3XBGZ5XANCNFSM6AAAAAAW6TF6GA

. You are receiving this because you commented.Message ID: @.***>

— Reply to this email directly, view it on GitHub <https://github.com/cmu-phil/tetrad/issues/1589#issuecomment-1509260668 , or unsubscribe < https://github.com/notifications/unsubscribe-auth/ACLFSR76OF2GZMRYBGALBJLXBG3MHANCNFSM6AAAAAAW6TF6GA

. You are receiving this because you authored the thread.Message ID: @.***>

— Reply to this email directly, view it on GitHub https://github.com/cmu-phil/tetrad/issues/1589#issuecomment-1509318425, or unsubscribe https://github.com/notifications/unsubscribe-auth/AD4Y3OIRUOBWCURHIS27B7DXBHAZRANCNFSM6AAAAAAW6TF6GA . You are receiving this because you commented.Message ID: @.***>

[jdramsey]

Plan.

The goal is to remove a smallest set of variables without which there are no singularities. When the user clicks "Find Next," the search proceeds from d = 0 up and checks all combinations of variables in sequence to find a combination, removing which (D'D)^-1 is nonsingular. If a combination is found, it is reported. If "Find Next" is not greyed out and the user clicks it again, the search continues until another combination is found, which is reported as a second option. And so on. If any combination can be found at a depth, this is reported, and the search stops and will not continue to the next depth. ("Find Next" will, in this case, be greyed out.)

The combinations found so far are given as a list of radio buttons, one of which must be selected as the variable set to remove.

This is all done in a "parameter editor" window like the DisretizationParamsEditor, and when the user selects "OK" the combination they've selected is removed from the data and the modified dataset is displayed.

[jdramsey]

Bryan suggested instead doing this to focus on singularity exceptions as they come up in algorithms. We used to do some of this, but perhaps something more systematic could be done for it across all algorithms.

[cg09]

What does the title of your email even mean?

On Sat, Jun 3, 2023 at 5:38 PM Joseph Ramsey @.***> wrote:

Bryan suggested instead doing this to focus on singularity exceptions as they come up in algorithms. We used to do some of this, but perhaps something more systematic could be done for it across all algorithms.

— Reply to this email directly, view it on GitHub https://github.com/cmu-phil/tetrad/issues/1589#issuecomment-1575207723, or unsubscribe https://github.com/notifications/unsubscribe-auth/AD4Y3OLB3Y5OAFWYXCLJ6C3XJOVF7ANCNFSM6AAAAAAW6TF6GA . You are receiving this because you commented.Message ID: @.***>

[jdramsey]

Oh you mean what issue is this in the issues list?

The problem is that if you give me a dataset there may be nonsingularities in the data. Mike had been figuring ut where these were by hand; I was going to automate that procedure. Bryan suggested instead that I focus on the nonsingularities (as I used to) by reporting the tests and scores where they were an issue.

[jdramsey]

Done, will be published with the next release.