search

FCP-INDI / C-PAC

Configurable Pipeline for the Analysis of Connectomes
https://fcp-indi.github.io/
GNU Lesser General Public License v3.0
64 stars 41 forks source link

Matrix is rank deficient #1093

Open dhinkris opened 5 years ago

dhinkris commented 5 years ago

Hi, I have an issue in calculating mdmr based cwas calculation. It would be great if you could help me with this.

I have 7 phenotypic variables and 19900 voxel-pairing for 60 subjects. I am trying to find cwas based on mdmr. Here I consider the X as the phenotypic variable (60x7) and the Y as the voxel-pairing(60, 19900). I am getting the “Exception: matrix is rank deficient (rank 60 vs cols 19900), whereas it runs for (60, >60) voxel-pairing with no significance.

I am lost at the moment. Please let me know where i am missing. Thanks

sgiavasis commented 4 years ago

Hi @dhinkris,

We'll be glad to assist. First, can you paste the full traceback of that error/exception? We'd like to see where it came from exactly if possible, a line number etc.

Also, if you are able to share it, it would also help to see the data and any configs you have used. If you cannot share this, then no worries.

Best, Steve

dhinkris commented 4 years ago

Hi @sgiavasis, Below is the piece of code I am trying to execute.

def test_mdmr():
    from CPAC.cwas.mdmr import mdmr
    from CPAC.cwas.cwas import calc_cwas
    import numpy as np
    X = np.genfromtxt(os.path.join(os.path.dirname(__file__), 'predictors_n59.csv'), delimiter=',')
    Y = np.genfromtxt(os.path.join(os.path.dirname(__file__), 'Ymat_n59.csv'), delimiter=',')
    print("Shape of X:", X.shape)
    print("Shape of Y:", Y.shape)
    X = X.reshape((X.shape[0], X.shape[1], 1))
    F_value, p_value = calc_cwas(X, Y, np.array([0], dtype=int), 1000, range(0, X.shape[1]))
    print(F_value, p_value)

And the traceback is : ('Shape of X:', (59, 9)) ('Shape of Y:', (59, 19900)) (9, 59, 59) Traceback (most recent call last): File "test_cwas.py", line 19, in test_mdmr() File "test_cwas.py", line 17, in test_mdmr F_value, p_value = calc_cwas(X, Y, np.array([0], dtype=int), 1000, range(0, X.shape[1])) File "/System/Volumes/Data/data/mril/users/all/mrdata/research/processed/CNMC/chd_r01/fetal/wip/rs/fetuses/projects/hlhs/analyses/grp_analysis_4_cwas/CPAC/cwas/cwas.py", line 66, in calc_cwas F_set, p_set = calc_mdmrs(D, regressor, regressor_selected_cols, permutations) File "/System/Volumes/Data/data/mril/users/all/mrdata/research/processed/CNMC/chd_r01/fetal/wip/rs/fetuses/projects/hlhs/analyses/grp_analysis_4_cwas/CPAC/cwas/cwas.py", line 44, in calc_mdmrs F_set, p_set = mdmr(D, regressor, cols, permutations) File "/System/Volumes/Data/data/mril/users/all/mrdata/research/processed/CNMC/chd_r01/fetal/wip/rs/fetuses/projects/hlhs/analyses/grp_analysis_4_cwas/CPAC/cwas/mdmr.py", line 74, in mdmr check_rank(X) File "/System/Volumes/Data/data/mril/users/all/mrdata/research/processed/CNMC/chd_r01/fetal/wip/rs/fetuses/projects/hlhs/analyses/grp_analysis_4_cwas/CPAC/cwas/mdmr.py", line 7, in check_rank raise Exception("matrix is rank deficient (rank %i vs cols %i)" % (rank, k)) Exception: matrix is rank deficient (rank 59 vs cols 19900)

Thank you.

dhinkris commented 4 years ago

Hi Steve, Did you get a chance to look into this issue.

Thank you.