GeoStat-Framework / PyKrige

Kriging Toolkit for Python
https://pykrige.readthedocs.io
BSD 3-Clause "New" or "Revised" License
741 stars 186 forks source link

Singular Matrix Error #160

Closed Marion-Nicco closed 3 years ago

Marion-Nicco commented 3 years ago

I am encountering this issue when trying to krige a dataset of 37,000 points (recorded using coordinates x, y and z and value V, euclidean). Unfortunately I can't share the dataset for reproducing the error, as it is confidential.

Here are the steps I took to troubleshoot: 1) Process all the data in smaller subsets of about 10,000 points. This runs fine and takes approx. 10hrs (ok3d only, not the execute command). I do know it is better to work with subsets but the error only appears when trying with the whole dataset (37k points) 2) Install the new pseudo-inv branch as suggested in #150 . I get the same error message (screenshot attached) however it appears that the dataset did not pass the new test implemented in core.py on line 775 (linal.lstsq) and instead went to line 777 (linalg.solve) 3) Change linalg.solve with linalg.lstsq in the loop on line 777. Same error.

I am starting to wonder if this is a size issue ? In #150 you suggest "decreasing the number of conditional points so the kriging matrix is invertible". What are conditional points? Is there a size limit to the matrix that makes it non-invertible?

I have an unrelated question that I can open another issue for if necessary: is it possible to reduce the lag distance taken into account when calculating the variogram (similar to maxlag in scikit gstat)? I found : core.py line 487 dmax=np.amax(d) . I tried to manually change it however when showing the variogram the max lag distance did not change and the bins were still calculated for the whole distance.

Any help is much appreciated, thank you!

image

MuellerSeb commented 3 years ago

Hey there,

if you are using the pseudo-inv branch, you have to set pseudo_inv=True in the OrdinaryKriging3D class. (like verbose=True in your case) Maybe that helps.

Variogram estimation will be updated with the v2 version. You can also try to use GSTools from the current develop branch, there you can setup the variogram estimation yourself, see here:

MuellerSeb commented 3 years ago

This should be fixed with v1.5.1 that will be released tomorrow. Closing for now. Feel free to re-open, if you have further questions.