There is a doubt： this modification will improve performance globally? sure it is slow in SVD.

[junphine]

class DataSet:

/**

• Creates a matrix from the data set, where each row represent a data
• point, and each column is one of the numeric example from the data set.
• 
  
• This matrix can be altered and will not effect any of the values in the data set.

• @return a matrix of the data points. */

  public Matrix getDataMatrix()
  {
      //modify by me Out of memory for text feathers
      int len = this.getNumNumericalVars();
      Vec first = getDataPoint(0).getNumericalValues();
      if(!first.isSparse()){
          DenseMatrix matrix = new DenseMatrix(this.getSampleSize(), len);
  
          for(int i = 0; i < getSampleSize(); i++)
          {
              Vec row = getDataPoint(i).getNumericalValues();
              for(int j = 0; j < row.length(); j++)
                  matrix.set(i, j, row.get(j));
          }
  
          return matrix;
      }
      else{           
          SparseVector[] rows = new SparseVector[this.getSampleSize()];
          for(int i = 0; i < getSampleSize(); i++)
          {
             Vec row = getDataPoint(i).getNumericalValues();
             rows[i] = (SparseVector)row.clone();
          }
          SparseMatrix matrix = new SparseMatrix(rows);
  
          return matrix;
      }
  }

[EdwardRaff]

Can you clarify what you are asking? This doesn't look related to the SVD code.

The SVD implementation in JSAT isn't designed for sparse matrices. A sparse friendly SVD is on my TODO list.

[junphine]

I modify the code， because memory overflow error occurred in the original version on PCA。 So I actually want to know PCA.fit(DataSet dataSet) performance.

_

[junphine]

This doesn't look related to the SVD code.

find the code GapStatistic related to SVD， But I haven't used it.

[EdwardRaff]

I don't understand this code suggestion and am closing it.

I'll make getDataMatrix return a sparse matrix, because that does make sense. But Neither the GapStatistic or SVD will be helped by this change.