sissa-data-science / DADApy

Distance-based Analysis of DAta-manifolds in python
https://dadapy.readthedocs.io/
Apache License 2.0
103 stars 17 forks source link

Problem with LSDI option on BMTI density estimation #138

Open alexdepremia opened 1 month ago

alexdepremia commented 1 month ago

Subject of the issue

When using delta_F_inv_cov="LSDI" while calling compute_density_BMTI, an error appears and the call crashes.

Your environment

colab notebook, python 3.10, direct installation of dadapy with !pip install dadapy provides "Successfully installed dadapy-0.3.1"

Steps to reproduce

X = np.genfromtxt(fname="./Fig1.dat") data = Data(coordinates=X) data.computedistances(maxk=1000, metric="euclidean") = data.compute_id_2NN(algorithm="base", mu_fraction=0.9, data_fraction=1) data.compute_kstar() data.compute_density_BMTI(delta_F_inv_cov="LSDI", comp_log_den_err=True, alpha=0.7)

Expected behaviour

Compute densities as in the case "delta_F_inv_cov=uncorr".

Actual behaviour

Crashes with the error:

/usr/local/lib/python3.10/dist-packages/dadapy/density_advanced.py in compute_density_BMTI(self, delta_F_inv_cov, comp_log_den_err, mem_efficient, alpha, log_den, log_den_err) 338 339 # define the likelihood covarince matrix --> 340 A, deltaFcum = self._get_BMTI_reg_linear_system(delta_F_inv_cov, alpha) 341 342 sec2 = time.time()

/usr/local/lib/python3.10/dist-packages/dadapy/density_advanced.py in _get_BMTI_reg_linear_system(self, delta_F_inv_cov, alpha) 381 ) 382 elif delta_F_inv_cov == "LSDI": --> 383 self.compute_deltaFs_inv_cross_covariance() 384 tmpvec = self.inv_deltaFs_cov 385

/usr/local/lib/python3.10/dist-packages/dadapy/density_advanced.py in compute_deltaFs_inv_cross_covariance(self, pearson_method) 251 sec = time.time() 252 # compute a diagonal approximation of the inverse of the cross-covariance matrix --> 253 self.inv_deltaFs_cov = cgr.return_deltaFs_inv_cross_covariance( 254 self.grads_var, 255 self.neigh_vector_diffs,

cython_grads.pyx in dadapy._cython.cython_grads.return_deltaFs_inv_cross_covariance()

ValueError: Buffer has wrong number of dimensions (expected 3, got 2)

charliematteo commented 1 month ago

I found the bug. It was a remainder of when we used only one name, "grads_var" and a flag to check whether grads_var was an array of vectors (variance per gradient component for each point i) or of matrices (autocovariance matrix of the gradient at each point i). Then we changed to grads_var and grads_covmat, two separate attributes of the class, but I forgot to change it in the specific function.

If you want, you can mark the issue as solved @alexdepremia

The numerics of the matrix inversion need to be checked I feel, but this is another issue :)