search

PantheonPlusSH0ES / DataRelease

44 stars 18 forks source link

Singular Covariance #2

Closed fsorrenti closed 2 years ago

fsorrenti commented 2 years ago

Hi everybody!

Thanks for your work. Looking at the covariance matrix "Pantheon+SH0ES_STAT+SYS.cov" and "Pantheon+SH0ES_STAT.cov", both of them seems to be singular. Are there any mistakes?

djbrout commented 2 years ago

They are invertible, so they shouldn't be singular. I suspect you're finding that they're singular because taking the determinant of a matrix filled with very small numbers can lead to floating point issues.

fsorrenti commented 2 years ago

You are perfectly right. Neither scipy nor numpy were able to invert the matrix giving "LinAlgError: Singular matrix".

I tried to do it using a Mathematica Notebook and it works.

Thanks for the attention and advises!

djbrout commented 2 years ago

fyi, this works for me

covf = open('../4_DISTANCES_AND_COVAR/Pantheon+SH0ES_STAT+SYS.cov').readlines()
covl = np.array(covf,dtype='float')
shape = covl[0]
covmat = covl[1:].reshape((int(shape),int(shape)))
print(np.linalg.inv(covmat))
fsorrenti commented 2 years ago

Thanks a lot! It works even for me. On the contrary, the followings (simpler ones) that I previously used give me the error: "LinAlgError: Singular matrix"

dimension=1701 input=np.genfromtxt('../4_DISTANCES_AND_COVAR/Pantheon+SH0ES_STAT+SYS.cov') matrix=np.zeros([dimension,dimension]) for j in range(0,dimension-1): for i in range(0,dimension-1): matrix[i,j]=input[dimension*j+i] np.linalg.inv(cov_plus)