Matrix is not full rank - error when there is zero variance in any state

[GoogleCodeExporter]

What steps will reproduce the problem?
1. use kmeans algorithm to learn initial hmm with OpdfMultiGaussianDistribution 
and two dimensional vectors
2. try to learn sequences where it somehow comes to states without variance in 
the kmeans algorithm. For example when you try to kmeans-learn an hmm with two 
states using one sequence with two observations ([ 0,136 0 ]; [ 9,595 0,5 ]; ).
Now the first observation will represent state 1 and the second observation 
will represent state 2. Both states have no variance in this case.
States without variance could also happen with more observations by random and 
this can not be predictable easily as it seems.

What is the expected output? What do you see instead?
it should also learn observations that end up in states without variance. 
States without variance produce a "Matrix is not full rank" error

Please provide any additional information below.

Used algorithms of Jahmm end up in a problem when somewhere no variance exists 
for a OpdfMultiGaussian function (mathematical problem). Zero variance is not 
illegal by itself (it can legaly occur in some cases. acutally they should only 
be zero in infinity) but used algorithms can't calculate with 0 as a variance 
value. Zero variance in one dimension would mean that this dimension kind of 
doesn't exist; it can not be calculated the distance (=probability) to the mean 
value of the gaussian curve in this dimension. Variance would not be zero (but 
probably very close as possible to zero) as soon as a little variance is 
existing (-> ">0")... Therefor we could set "d" (in SimpleMatrix.java:260) to 
"0.00000000000001" for example if it is already equal to 0 at this position. Of 
course this does mean that we manipulate the gaussian distribution, but since 
this is about statistics an probability this manipulation is VERY little. 
Depending on your system and your needs this could be a good and simple 
workaround for the "Matrix is not full rank" error.

Go to SimpleMatrix.java:260 and add:
        if(d==0){
            d = 0.0000000001;
        }

kind regards,
Ben

Original issue reported on code.google.com by vamos.be...@gmail.com on 9 Jun 2010 at 1:13

[GoogleCodeExporter]

Hi ben,

Thanks for clarifying this (and providing a solution). One question though:

I cannot seem to figure out where to place the provided exactly. The method 
around line 260 at SimpleMatrix is:

    static double determinantCholesky(double[][] l)
    {
        if (!isSquare(l))
            throw new IllegalArgumentException("Matrix is not square");

---------do i place code here????----------------

        double d = 1.;
        for (int i = 0; i < nbRows(l); i++)
            d *= l[i][i];

        return d * d;
    }

I hope you can still remember what you have done, it seems a while ago..

Thank you 

Jorrit

Original comment by jorre...@gmail.com on 18 Jul 2012 at 7:28