Mismatches to Markus's data

[martin-ueding]

It seems that for P = 001 and A1 it works pretty well: comparison-001-A1-1.pdf

Here we can try a larger set of q values for a larger correlator matrix, but it seems rather legit. The factor 2 that they deviate is already fixed in the code, but not in that plot.

But for P = 100 and A1 there still mismatches: comparison-100-A1.pdf

So something is still off.

[martin-ueding]

We believe that the only issue was that I was comparing to the reference frame all the time, therefore it did not work. We fixed that yesterday, and at least for cutoff = 1 this worked out. I will have to run the projection for higher cutoffs (taking more walltime) and compare a larger correlator matrix now.

[urbach]

very good, I think this test will be very valuable!

[martin-ueding]

For P²=0 and the T1u irrep I now also get agreement, albeit with a constant factor still. Somehow the normalization does not match, but that is not really an issue because each correlator matrix seems to have one consistent scale factor.

all.pdf

Somehow my E irrep does not couple to anything, but Markus has coupling there. I will need to investigate this further.

[martin-ueding]

For P = (1, 1, 1) and the E irrep I get a missmatch with either the first or the second column of the irrep.

comparison-111-E.pdf

The P = (1, 1, 1) with A1 works fine, though. So it seems to be that particular irrep which does not work.

[martin-ueding]

In the P = (0, 0, 1) frame the E does not work either: comparison-001-E.pdf

So it seems to be a systematic problem with the E irrep.

[marcuspetschlies]

Hi Martin,

    quick question, are you comparing up to change of basis vectors ?

Thanks, best regards,

Marcus

On 23.04.19 15:10, Martin Ueding wrote:

In the P = (0, 0, 1) frame the E does not work either: comparison-001-E.pdf https://github.com/HISKP-LQCD/sLapH-projection-NG/files/3107662/comparison-001-E.pdf

So it seems to be a systematic problem with the E irrep.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/HISKP-LQCD/sLapH-projection-NG/issues/8#issuecomment-485795934, or mute the thread https://github.com/notifications/unsubscribe-auth/ADDSFIIB4RNVWPNMVLN52H3PR4DENANCNFSM4HE626HQ.

[martin-ueding]

Since @maowerner gave me those basis vectors I expect them to match his.

[marcuspetschlies]

I am not sure, what exactly is done in your code now, so maybe the question is ill-placed; do you compare re and im ( in case both non-zero ) ?

[martin-ueding]

Using Mathematica I compute the projection coefficients. These are complex numbers that tell me which correlators (complex valued) from the contraction code I need to combine. One of these coefficients looks like this:

{
    "datasetname": "C4cB_uuuu_p000.d000.g5_p000.d000.g5_p-111.d000.g5_p1-1-1.d000.g5",
    "re": 0.5,
    "im": 0,
    "conj": false
}

I need to take that particular dataset from the HDF5 file, not apply complex conjugation to it, multiply it with 0.5 + 0 i. I have a bunch of these coefficients for each entry in the correlator matrix. After I have made the sum I expect the signal to be in the real part and therefore just discard the imaginary part.

Therefore I only compare real parts. @maowerner just gave me the real parts from his code as well.

[maowerner]

Could you once again show the ratio plots between my and your results?

[maowerner]

The other thing I would like to see is a sanity check whether your result is consistent with your results for p = 010, 100, etc.

[martin-ueding]

I have re-run projection and actual number crunching, the mismatch in the E irreps is still there. I'll then proceed further with above sanity check.

[martin-ueding]

For instance d² = 1 with the A1 looks like this for various momenta (colors) contributing the same signals on the diagonal and just similar noise on the other terms:

I do not understand where the ones in x direction are, though. I'll have to investigate this.

With similar problems there is a partial result for d² = 2 in the A1:

[martin-ueding]

Just an administrative issue, nothing to do with the data. So here it is! The removal of duplicate columns does not seem to work yet, the numerical data is not exactly the same. I will have to think about a better way to remove duplicate columns than just checking for exact replication of C(t = 0).

Except for that duplicate columns issue I'd say that within each moving frame and irrep the various relative momenta and irrep rows and columns are compatible with each other.

[maowerner]

There are some off-diagonal contributions which seem a bit suspect. A prime example is d²=1 A1 002 -> 001. It looks like d = +- e_y have a different sign compared to the rest. Something similar might happen for d²=1 E 0-11 -> -101 which seem to split up into positive and negative d. But seeing as the diagonal elements all are degenerate, I think this is at least very close to correct

Which basis are the Wigner-D matrices in?

[martin-ueding]

I have rewritten the code for removing duplicate columns, but it still produces plots like this one here:

We can have a look at the C(t = 0) values as a matrix:

           [,1]        [,2]       [,3]        [,4]        [,5]       [,6]
[1,] -3510.7996    -10.8356  1767.2536      2.2812      8.5544  1743.5460
[2,]    -4.5589 -12299.2061   -10.6474   6170.0876   6129.1184    15.2063
[3,]  1743.5460      2.2812 -3510.7996      8.5544    -10.8356  1767.2536
[4,]   -10.6474   6129.1184    15.2063 -12299.2061   6170.0876    -4.5589
[5,]    15.2063   6170.0876    -4.5589   6129.1184 -12299.2061   -10.6474
[6,]  1767.2536      8.5544  1743.5460    -10.8356      2.2812 -3510.7996

This resembles the image that we see above. But strikingly this matrix is not symmetric. Subtracting the transpose gives us the following:

         [,1]     [,2]     [,3]     [,4]     [,5]     [,6]
[1,]   0.0000  -6.2767  23.7076  12.9286  -6.6519 -23.7076
[2,]   6.2767   0.0000 -12.9286  40.9692 -40.9692   6.6519
[3,] -23.7076  12.9286   0.0000  -6.6519  -6.2767  23.7076
[4,] -12.9286 -40.9692   6.6519   0.0000  40.9692   6.2767
[5,]   6.6519  40.9692   6.2767 -40.9692   0.0000 -12.9286
[6,]  23.7076  -6.6519 -23.7076  -6.2767  12.9286   0.0000

The diagonal is fine, but the other elements show deviations. Dividing this by the original matrix elements, we obtain the relative non-symmetricness:

            [,1]         [,2]        [,3]         [,4]         [,5]        [,6]
[1,]  0.00000000  0.579266492  0.01341494  5.667455725 -0.777599832 -0.01359735
[2,] -1.37680142  0.000000000  1.21424949  0.006639971 -0.006684354  0.43744369
[3,] -0.01359735  5.667455725  0.00000000 -0.777599832  0.579266492  0.01341494
[4,]  1.21424949 -0.006684354  0.43744369  0.000000000  0.006639971 -1.37680142
[5,]  0.43744369  0.006639971 -1.37680142 -0.006684354  0.000000000  1.21424949
[6,]  0.01341494 -0.777599832 -0.01359735  0.579266492  5.667455725  0.00000000

Some of them are quite large. I would have expected the correlator matrix to be exactly symmetric. Or is that only true in the gauge average? If so, how do I identify duplicate columns and rows? If not, what could cause the slight deviations?

[martin-ueding]

Which basis are the Wigner-D matrices in?

Does not matter at this point as they are only 1 for all the values that we put in.

[martin-ueding]

A prime example is d²=1 A1 002 -> 001. It looks like d = +- e_y have a different sign compared to the rest.

Really? It seems that all the y ones (yellow, pink) are scattered at the upper and lower branch, just like the other colors.

Something similar might happen for d²=1 E 0-11 -> -101 which seem to split up into positive and negative d.

Yeah, that is indeed peculiar!

[maowerner]

Could you post the same matrix for the imaginary parts?

[martin-ueding]

What I could do is take the imaginary part instead of the real part after applying the projection coefficients. Is that what you have in mind?

[maowerner]

Yes exactly. I would like to see wether a comlex conjugate or something could be responsible for this deviation of certain moving frames. Then you would see the same difference in the imaginary part as well. The signal looks like you could explain it by noise, but in my opininion it is to correlated for comfort

[martin-ueding]

Taking the imaginary part instead of the real part after the projection just seems to give nonsense for every single correlator matrix: imaginary.pdf

[martin-ueding]

To summarize the status: Using the real part gives most (real.pdf) correlator matrices exactly, there is a scaling factor in the d² = 0 T1u and none of the E irreps work out correctly.

For the E irreps the different equivalent momenta give the same results for the diagonal elements. The off diagonal elements sometimes show a significant different global sign for the correlation function, which is weird.

What could I look at next?

[maowerner]

Could you compare your data with Re(target) - Im(target)? Or even better Re(target) - Re(actual) with Im(target)? Or is this the target real part in imaginary.pdf?

If I see that correctly, in imaginary.pdf you compared the target real part with the actual imaginary part. Could you take the imaginary part for both of them to see wether there is a deviation in the opposite direction? I want to rule out, that some phase flipped a contribution to the imaginary part.

[martin-ueding]

If I see that correctly, in imaginary.pdf you compared the target real part with the actual imaginary part.

Correct!

Could you take the imaginary part for both of them to see wether there is a deviation in the opposite direction?

I do not have the imaginary part for your data, the correlator matrices that you have written out just contain the real part.

[maowerner]

I do not have the imaginary part for your data, the correlator matrices that you have written out just contain the real part.

What have I done -.- Which Ensemble / gauge configs are you looking at?

[martin-ueding]

What have I done -.-

I guess a good mixture of YAGNI and Pareto principle.

Which Ensemble / gauge configs are you looking at?

I am looking at B35.32 in configuration 2552.

[martin-ueding]

I have re-run the projections with the latest trustworthy version of my projection code. There was some momentum sign flip that that pretty much broke everything. For some reason not all irreps finish in a reasonable time, some jobs are still running.

For the ones that have finished I have generated the comparison plots all-compare.pdf. The E irreps are all broken, so here are the ones with P²=1 all-P1-E.pdf and P²=3 all-P3-E.pdf. I was hoping that one could see some clear mix-up or so.

All irreps except the E just seem to work perfectly. And although I do not have the plot here, the T1u worked at some point and that is a multi-dimensional irrep. So in principle some non-scalar irreps work and it is just the E irrep that does not work. Skimming through this issue tells me that we have tried both columns already. So I'll need to dig into this again.

[martin-ueding]

The E irrep is the only two-dimensional one. It is not the odd momentum as for P²=1 and P²=3 the A and B irreps do work out. Is there something else special about the E irrep?

[maowerner]

The E irrep is the only one with "mixed" coefficients. In all other irreps the are either real or imaginary, but E has these (1+i) factors.

You could look at the difference between actual and target if you have not done that already. Does it look like a correlator? Basically there are more terms that enter.

The other very powerful test is whether both rows yield the same correlator. This helped me a lot to find missing complex conjugations and the like.

[martin-ueding]

The only irrep having imaginary coefficients is the T1u. All others have purely real coefficients. The T1u works, at least last time I checked. So just adding a complex conjugation to the representation would not alter anything except the T1u and all the E irreps.

I will first get the T1u back to work.

[kostrzewa]

The only irrep having imaginary coefficients is the T1u.

Are you sure about that? I'm really out of this topic at this point but a quick glance at the papers suggests that the E irrep also has mixed coefficients. (at least in the basis of arXiv:1206.4141)

[martin-ueding]

What I meant to say is that the only other except the E with complex coefficients is the T1u. So you're right.

[martin-ueding]

Projection for the T1u takes forever because the subgroup (full group) is large compared to the other moving frames. I have done it with a momentum cutoff of 1 quickly now, and that still matches up to a factor.

comparison-000-T1u.pdf

I need to take the second irrep column (or row?), though in order to get the right momenta coupling. If I take the first one I get Y coupling, the second ones gives me Z (as Markus has) and the third gives me X.

[marcuspetschlies]

Hi Markus & Martin,

    let me know if going back to some independent comparison would be helpful. I have Markus' irrep matrices and can produce the subduction coeffs. for step-by-step comparison of which correlators are combined how.

With best regards,

Marcus

On 30.07.19 21:11, Markus Werner wrote:

The E irrep is the only one with "mixed" coefficients. In all other irreps the are either real or imaginary, but E has these (1+i) factors.

You could look at the difference between actual and target if you have not done that already. Does it look like a correlator? Basically there are more terms that enter.

The other very powerful test is whether both rows yield the same correlator. This helped me a lot to find missing complex conjugations and the like.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/HISKP-LQCD/sLapH-projection-NG/issues/8?email_source=notifications&email_token=ADDSFIOXNN7SZEDISOUHNIDQCCG5RA5CNFSM4HE626H2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3E75KI#issuecomment-516554409, or mute the thread https://github.com/notifications/unsubscribe-auth/ADDSFIO5WZNBUI7R56Y6OL3QCCG5RANCNFSM4HE626HQ.

[martin-ueding]

I have just taken out the complex conjugation where the representation matrix is used.

diff --git a/Wolfram_Language/sLapHProjection.wl b/Wolfram_Language/sLapHProjection.wl
index 421f33f..0744b0e 100644
--- a/Wolfram_Language/sLapHProjection.wl
+++ b/Wolfram_Language/sLapHProjection.wl
@@ -164,7 +164,7 @@ MakeGroupSum[irrep_, irrepRow_, irrepCol_, momentapi_, spinsJi_, spinsMi_] := Mo
       name = Keys @ #;
       values = Values @ #;
       eulerG = EulerAnglesParityAssoc[][[Key @ name]];
-      Conjugate[values[[Key @ {irrepRow, irrepCol}]]] *
+      values[[Key @ {irrepRow, irrepCol}]] *
       MakeMultiOperator[momentapi, eulerG, spinsJi, spinsMi]] &,
     IrrepDGammaAssoc[][[Key @ MomentumRef @ Total @ momentapi]][[Key @ irrep]]];
   Plus @@ groupSummands / Length[groupSummands]];

At least for q=010 this does not change anything. I will now try higher relative momenta and also take a look into the E irreps. As all the coefficients are real for all other irreps they must not change.

[marcuspetschlies]

Maybe you could write out in text format a table like

irrep --- row source --- row sink --- coeff --- correlator

and correlator specifies which data set goes in, like Op_snk - Op_src

Then I can go ahead and produce the same table.

Once we agree on the input, we can make sure, that  what is written is also being used etc..

Best regards,

Marcus

Am 31.07.19 um 12:29 schrieb Martin Ueding:

I have just taken out the complex conjugation where the representation matrix is used.

diff --git a/Wolfram_Language/sLapHProjection.wl b/Wolfram_Language/sLapHProjection.wl index 421f33f..0744b0e 100644 --- a/Wolfram_Language/sLapHProjection.wl +++ b/WolframLanguage/sLapHProjection.wl @@ -164,7 +164,7 @@ MakeGroupSum[irrep, irrepRow, irrepCol, momentapi, spinsJi, spinsMi_] := Mo name = Keys @ #; values = Values @ #; eulerG = EulerAnglesParityAssoc[][[Key @ name]];

• Conjugate[values[[Key @ {irrepRow, irrepCol}]]] *
• values[[Key @ {irrepRow, irrepCol}]] * MakeMultiOperator[momentapi, eulerG, spinsJi, spinsMi]] &, IrrepDGammaAssoc[][[Key @ MomentumRef @ Total @ momentapi]][[Key @ irrep]]]; Plus @@ groupSummands / Length[groupSummands]];

At least for q=010 this does not change anything. I will now try higher relative momenta and also take a look into the E irreps. As all the coefficients are real for all other irreps they must not change.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/HISKP-LQCD/sLapH-projection-NG/issues/8?email_source=notifications&email_token=ADDSFIMJFPNCL37LYVWWJX3QCFSQHA5CNFSM4HE626H2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3G2DTQ#issuecomment-516792782, or mute the thread https://github.com/notifications/unsubscribe-auth/ADDSFIPYOT2UMOI2Q4WEKE3QCFSQHANCNFSM4HE626HQ.

[kostrzewa]

@marcuspetschlies @martin-ueding When you guys make this comparison, don't forget that in the sLapH contraction code, the momentum phase convention is that exp(-ipx) is applied at the source, while exp(iqx) is applied at the sink (IIRC), correct @martin-ueding?

[martin-ueding]

The performance of the code is horrible, even after days it is still working. In order to get somewhere I have now just computed it for a very limited amount of relative momenta. So on d = (0, 0, 1) in the E irrep I see that the little change of omitting the complex conjugation in the irrep element does not magically fix everything for one particular q.

@marcuspetschlies: I have the data in the following form:

I could massage this into a flat table if that would be more convenient. Then all these levels would become columns and then all the summands would be just rows in it. Which would you prefer?

[marcuspetschlies]

Let me have a look at it this way.

How did you make the arrow labels?

On 07.08.19 15:26, Martin Ueding wrote:

The performance of the code is horrible, even after days it is still working. In order to get somewhere I have now just computed it for a very limited amount of relative momenta. So on |d = (0, 0, 1)| in the E irrep I see that the little change of omitting the complex conjugation in the irrep element does not magically fix everything for one particular |q|.

@marcuspetschlies https://github.com/marcuspetschlies: I have the data in the following form:

Auswahl_004 https://user-images.githubusercontent.com/976924/62626378-8d7aa900-b927-11e9-820b-1610a0d8ce21.png

I could massage this into a flat table if that would be more convenient. Then all these levels would become columns and then all the summands would be just rows in it. Which would you prefer?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/HISKP-LQCD/sLapH-projection-NG/issues/8?email_source=notifications&email_token=ADDSFII6BQCFO2RZXVU4XELQDLEOZA5CNFSM4HE626H2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3YMNQI#issuecomment-519096001, or mute the thread https://github.com/notifications/unsubscribe-auth/ADDSFILMA3MNHRMSIOARRVLQDLEOZANCNFSM4HE626HQ.

[martin-ueding]

Having identified the performance problem to be the new version of Mathematica, I will now submit the jobs to produce the most data I can.

The screenshot was annotated with Shutter.

[martin-ueding]

A bunch of the projections have been running through now, here are the ones that are finished so far: gevp-rho-partial.tar.gz

[martin-ueding]

Also I have run the numerical projections and find agreement except for the E irreps. The T1u has not finished projecting yet. all.pdf

[martin-ueding]

@marcuspetschlies: All of the projections except the 000-Eg, which does not couple anyway, are now done:

gevp-rho-most.tar.gz

[martin-ueding]

I've now exposed all the irrep rows and columns in the comparison notebook as well. You can have a look at this if you excuse the P² = 0 with T1u for a brief moment.

Comparison_with_Markus_Werner.pdf

The problem is the visualization for the 3 dimensional irreps. As you can see there are only six shapes in the default palette and therefore not all combinations are visible:

The issue is that Markus's data does not really expose the irrep column \beta, he just takes a weighted sum. And the weight is in a way that it picks out just one of the columns, though I do not know for certain which ones it is going to be. Therefore I denote these as row.col with 1.NA.

I can exchange color and shape, this makes it easy to resolve everything in terms of palettes.

As we humans notice color first and shape second this plot is much harder to read. With a bit of squinting one can figure out that this matches pretty well and that there is just a normalization issue going on.

I think that I like the coloring better, this way in the E one can easily see that the different rows and columns do not make a large difference but the physical signal is similar.

I'll try to split the palette and come up with something better.

[martin-ueding]

Now red to green shows the actual data, blue to purple shows the target data. This way it splits into two groups rather quickly and it has consistent colors across all the plots.

[martin-ueding]

Using this new visualization we get the following notebook: Comparison_with_Markus_Werner.pdf

It seems that the E irreps in P² = 1 and P² = 4 describe the same physics. I wonder where there P² = 3 E irreps are. Also in the P² = 0 T1u there is this scaling factor and the relative momentum 001 missing. There are a few more details that I need to figure out.

[martin-ueding]

For some reason the off-diagonal terms do not show up, I am not not sure what this means.

[martin-ueding]

If you use [\d.] as a regular expression for decimals one should not be surprise that stuff with minus signs is just going to drop out.

Now these are the only irreps that occur in my data but not in Markus's:

total_momentum_str	irrep	config_number
000	T2u	2552
00-1	B1	2552
001	B1	2552
00-1	B2	2552
001	B2	2552
0-10	B1	2552
010	B1	2552
0-10	B2	2552
010	B2	2552
-100	B1	2552
100	B1	2552
-100	B2	2552
100	B2	2552

For the T2u I know that it does not couple to into the P-wave and therefore is not interesting. I presume that these remaining ones are also of no interest for the rho ignoring higher partial waves?

This is the most comprehensive comparison that I have up to this point: Comparison_with_Markus_Werner.pdf