using a covariance matrix

[pbakalov]

Is using a covariance matrix fully supported at this stage?

When I try and use a covariance matrix by setting COVARIANCE_MATRIX in the input file, the simulation ignores it and runs as if COVARIANCE_MATRIX is not set.

This is because COVARIANCE_MATRIX is not defined in maxent_simulation.cpp.

After adding the following line to maxent_simulation.cpp:

 p.define<std::string>("COVARIANCE_MATRIX","","name of covariance matrix file");

the covariance matrix file is read, but the results of the analytic continuation aren't encouraging.

Is this because there's something wrong with my covariance matrix, or is Maxent not ready to handle this case at this point?

[egull]

@ryanlevy would you have time/capacity to have a look at this?

[ryanlevy]

Yeah I can look into this! Just a note, the covariance matrix input was not rigorously tested (as I wasn't able to procure a good dataset to write a test), but theoretically works.

Silly question, did you check that the input was in the proper format? I think it's rather finicky. Specifically if I recall correctly the input Green's function data needs to have error bars on the input for instance

[egull]

@pbakalov could you please just upload your input data and parameter files?

[pbakalov]

Hi, I'm attaching the three input files (imaginary time GF & covariance matrix & parameter file).

The GF corresponds to the DMFT solution of a Hubbard model on a cubic lattice with n.n. hopping t =1 in 3D at inverse temp \beta = 30 and U = 9.

The covariance matrix is based on 1500 bins for each \tau_i (i = 1, ... , 300).

With this particular dataset, maxent says

# Ignoring singular eigenvalues (0-248 out of 300)

With another data set with the same number of bins but with the bin size 10 times smaller (i.e. each bin contains a number of measurements 10 times smaller than in the first data set), I get

# Ignoring singular eigenvalues (0-37 out of 300)

I think this behaviour can be traced down to the cutoff in the eigenvalue spectrum, which is currently fixed at 1e-10.

Unless I'm missing something, I think a fixed (pre-set) cutoff of this kind can lead to discarding good eigenvalues, as the size of the eigenvalues of the covariance matrix can vary with the quality of the MC data and the number of bins (e.g. a larger bin size at fixed number of bins would normally lead to smaller eigenvalues). I'm not sure I have a suggestion how to go about it in a better way, except look at the spectrum and manually select the cutoff.

Both cases above lead to a nonsense spectrum.

Decreasing the cutoff to 1e-12 allows all eigenvalues to be retained (the eigenvalue spectrum looks continuous, i.e. no sharp break as one would expect for a small number of bins), but the minimiser crashes:.

alpha it: 51    Caught Exception Dynamic exception type: std::logic_error
std::exception::what: Q=NaN, something went wrong

Decreasing the number of real frequencies to 300 leads to a crash at alpha it ~ 30.

Plotting the spex.dat file shows that the spectrum makes some sense (the typical Hubbard three-peak structure) only at the largest two values of alpha (20 and ~17).

G_tau.txt time.param.txt cov_matrix_imtime.txt

[pbakalov]

I think I found (a big part of) the problem: the square root was missing in the line where sigma_ is set from the eigenvalues of the covariance matrix:

sigma_[i] = std::abs(var(new_ndat_+i))/static_cast<double>(p["NORM"]);

should be

sigma_[i] = std::sqrt(std::abs(var(new_ndat_+i)))/static_cast<double>(p["NORM"]);

Changing this line (in addition to the cutoff) leads to a reasonable spectrum with the above data.

I submitted a pull request

[egull]

I think you're right, especially in light of the comment on line 221. @ryanlevy please have a look.

[ryanlevy]

Just for documenting: looks like Xi removed the square root in 24e17199096cafc2eb06718629a6c94670f61d21, I'm not sure why. But it does mean that V1 has this bug.

[egull]

@xichenli , do you remember what you were doing at the time?

[xichenli]

I am sorry I can’t remember doing it on purpose. This is very likely a bug.

On May 25, 2018, at 3:26 PM, Emanuel Gull notifications@github.com wrote:

@xichenli https://github.com/xichenli , do you remember what you were doing at the time?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/CQMP/Maxent/issues/39#issuecomment-392157203, or mute the thread https://github.com/notifications/unsubscribe-auth/AHjB5VkQs-pczY249eq29LjFIjfLYgwSks5t2FrogaJpZM4UEyPo.

[egull]

Thank you very much for fixing this!!!

Please verify that it works for you and close the bug.

[pbakalov]

seems fine now!