search

flav-io / flavio

A Python package for flavour physics phenomenology in the Standard model and beyond
http://flav-io.github.io/
MIT License
71 stars 62 forks source link

Wrong B->D*lnu branching ratio #96

Closed DavidMStraub closed 5 years ago

DavidMStraub commented 5 years ago

The new form factors introduced in v1.6 must have some problem - the B->D*lnu branching ratio is too large by a factor of 4!

DavidMStraub commented 5 years ago

Correction: this was not yet released in 1.6 but is just part of master, fortunately.

DavidMStraub commented 5 years ago

@dvandyk @MartinJung1978

When implementing your new CLN form factors, something went terribly wrong. While @dvandyk and I carefully wrote a unit test and made it pass, so I'm very confident the implementation is correct, it seems the parameters are totally off. Indeed changing back to the old (2018) parameters the FFs are sane again.

So it seems something went wrong when translating the samples to parameter values. I do assume them to be a multivariate Gaussian, but I doubt this is the problem. I rather suspect that some parameters are interchanged. Can you please help us debug this?

This is the notebook I used to translate your samples to flavio parameters: https://gist.github.com/DavidMStraub/778f8acc2261733b4aeeb21531e85195 then copied to here: https://github.com/flav-io/flavio/blob/1fefe2161be7c3ca4a21614378ade0fe2eaccc9f/flavio/data/parameters_correlated.yml#L406-L454

Thanks in advance.

dvandyk commented 5 years ago

Hi @DavidMStraub

your "central" values are very different from the best fit point shown in table II of the paper by Martin, Marzia and me. In particular your values for CLN c_xi and to less extent CLN xi3 are numerically important.

Could you try set the central values as follows:

  CLN c_xi  0.94
  CLN xi3 -3.29

Contrary to you, I surmise that your assumption of Gaussian uncertainties is probably what causes the difference...

dvandyk commented 5 years ago

In addition, here is the list of parameters in order from my fit script:

    --scan       B(*)->D(*)::xi'(1)@HQET   -2.5000e+00   -0.5000e+00  --prior flat
    --scan      B(*)->D(*)::xi''(1)@HQET    0.5000e+00   +5.5000e+00  --prior flat
    --scan     B(*)->D(*)::xi'''(1)@HQET   -1.2000e+01   +0.5000e+00  --prior flat
    --fix     B(*)->D(*)::xi''''(1)@HQET    0.0
    --fix    B(*)->D(*)::xi'''''(1)@HQET    0.0

    --scan     B(*)->D(*)::chi_2(1)@HQET   -3.0000e-01   +3.0000e-01  --prior flat
    --scan    B(*)->D(*)::chi_2'(1)@HQET   -7.5000e-02   +7.5000e-02  --prior flat
    --scan   B(*)->D(*)::chi_2''(1)@HQET   -1.0000e+00   +1.0000e+00  --prior flat

    --scan    B(*)->D(*)::chi_3'(1)@HQET   -3.0000e-01   +3.0000e-01  --prior flat
    --scan   B(*)->D(*)::chi_3''(1)@HQET   -1.0000e+00   +1.0000e+00  --prior flat

    --scan       B(*)->D(*)::eta(1)@HQET   -0.5000e+00   +1.5000e+00  --prior flat
    --scan      B(*)->D(*)::eta'(1)@HQET   -7.5000e-01   +7.5000e-01  --prior flat
    --scan     B(*)->D(*)::eta''(1)@HQET   -2.0000e+00   +3.0000e+00  --prior flat

    --scan       B(*)->D(*)::l_1(1)@HQET   -2.0000e+01   +2.0000e+01  --prior flat
    --scan      B(*)->D(*)::l_1'(1)@HQET   -3.0000e+01   +3.0000e+01  --prior flat

    --scan       B(*)->D(*)::l_2(1)@HQET   -2.0000e+01   +2.0000e+01  --prior flat
    --scan      B(*)->D(*)::l_2'(1)@HQET   -3.0000e+01   +3.0000e+01  --prior flat

    --scan       B(*)->D(*)::l_3(1)@HQET   -2.0000e+01   +2.0000e+01  --prior flat
    --scan      B(*)->D(*)::l_3'(1)@HQET   -3.0000e+01   +3.0000e+01  --prior flat

    --scan       B(*)->D(*)::l_4(1)@HQET   -2.0000e+01   +2.0000e+01  --prior flat
    --scan      B(*)->D(*)::l_4'(1)@HQET   -3.0000e+01   +3.0000e+01  --prior flat

    --scan       B(*)->D(*)::l_5(1)@HQET   -2.0000e+01   +2.0000e+01  --prior flat
    --scan      B(*)->D(*)::l_5'(1)@HQET   -3.0000e+01   +3.0000e+01  --prior flat

    --scan       B(*)->D(*)::l_6(1)@HQET   -2.0000e+01   +2.0000e+01  --prior flat
    --scan      B(*)->D(*)::l_6'(1)@HQET   -3.0000e+01   +3.0000e+01  --prior flat
dvandyk commented 5 years ago

Sorry for the piecemeal replies. I just had a chance to compare the lists. The order seems to be fine, so interchanging parameters seems not likely.

MartinJung1978 commented 5 years ago

And it's not by any chance rho2 vs -xi' or c vs x''/2 I guess?

Just making sure... M.

Danny van Dyk notifications@github.com schrieb am Mo., 18. Nov. 2019, 13:49:

Sorry for the piecemeal replies. I just had a chance to compare the lists. The order seems to be fine, so interchanging parameters seems not likely.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/flav-io/flavio/issues/96?email_source=notifications&email_token=AFRTFT7Y4WXLTSQCW5SSRL3QUKFPBA5CNFSM4JOKCTQ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEEKKNSI#issuecomment-555001545, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFRTFT2FESTPEGXRFLPYKKLQUKFPBANCNFSM4JOKCTQQ .

DavidMStraub commented 5 years ago

Contrary to you, I surmise that your assumption of Gaussian uncertainties is probably what causes the difference...

If it's true that taking the mean of each parameter from the samples gives a form factor that is off by a factor of 2-3, that would be quite worrisome, wouldn't it?

DavidMStraub commented 5 years ago

And it's not by any chance rho2 vs -xi' or c vs x''/2 I guess?

Possible; I did this in the notebook to account for it:

data = samples[:,:23]
data[:, 0] = -data[:, 0]  # xi' -> rho
data[:, 1] = 1/2 * data[:, 1]  # xi'' -> c

Correct?

dvandyk commented 5 years ago

Contrary to you, I surmise that your assumption of Gaussian uncertainties is probably what causes the difference...

If it's true that taking the mean of each parameter from the samples gives a form factor that is off by a factor of 2-3, that would be quite worrisome, wouldn't it?

David, this is not worrisome but rather expected, due to the role of the unitarity bound in the fit. It leads to a strongly non-gaussian posterior, which we emphasized also in the paper:

Uncertainty ranges presented here are meant for illustrative purpose only, and should not be interpreted as standard deviations due to non-Gaussianity of the joint posterior.

DavidMStraub commented 5 years ago

I'm talking about central values, not uncertainties...

talismanbrandi commented 5 years ago

Hi everyone,

this is not my place to speak but we had faced a similar problem in HEPfit in another context. As Danny pointed out, non-Gaussianity can lead to this problem, even more so if the parameters are correlated. The way to see this is that the means taken from the distributions of a set of parameters are not a correlated set. In the case of non-Gaussian distributions this can be made even worse leading to un-realistic values of the function.

In our famous h_lambda parameterization, if one takes the means of the distributions of the 16 parameters one will get a value of the branching fraction that will be very far from the fit value since the h_lambda set has a non-Gaussian distribution and is highly correlated.

hope this helps, ciao Ayan

On Nov 18, 2019, at 18:50, David Straub notifications@github.com wrote:

Contrary to you, I surmise that your assumption of Gaussian uncertainties is probably what causes the difference...

If it's true that taking the mean of each parameter from the samples gives a form factor that is off by a factor of 2-3, that would be quite worrisome, wouldn't it?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/flav-io/flavio/issues/96?email_source=notifications&email_token=AARPQ23AAWKBT7LHK3S6NI3QUKJBPA5CNFSM4JOKCTQ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEEKM6BQ#issuecomment-555011846, or unsubscribe https://github.com/notifications/unsubscribe-auth/AARPQ2436ZY3IW6IHZQJJM3QUKJBPANCNFSM4JOKCTQQ.

-- Ayan Paul

Office:

DESY, Hamburg &
Institut für Physik, Humboldt Universität zu Berlin
Newtonstrße 15 12489 Berlin, Germany Ph: (+49) 030 2093-7877

Residence:

Weichselstraße 31 Vod. 2 OG. Mitte 10247 Berlin, BE Germany Ph: (+39) 33480 94805

dvandyk commented 5 years ago

I'm talking about central values, not uncertainties...

David, the best-fit point stays as it is. It's just the for non-gaussian posterior the mean and the mode of the distribution do not coincide anymore. So using the mean (as the average value) will take you far away from the mode (the most likely value).

MartinJung1978 commented 5 years ago

Hi,

And it's not by any chance rho2 vs -xi' or c vs x''/2 I guess?

Possible; I did this in the notebook https://gist.github.com/DavidMStraub/778f8acc2261733b4aeeb21531e85195 to account for it:

data= samples[:,:23] data[:,0]= -data[:,0]# xi' -> rho data[:,1]= 1/2 * data[:,1]# xi'' -> c

Since I don't know which side of data[:, 1] is xi'' and which is c, it's either correct or introduces a factor 4 :-)

The relation should be 1/2 xi''(1) = c.

M.

Correct?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/flav-io/flavio/issues/96?email_source=notifications&email_token=AFRTFT4FQ53G4ZTEFJC3SEDQUKJRHA5CNFSM4JOKCTQ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEEKNLFQ#issuecomment-555013526, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFRTFTYJGGBM5BRQA6DWRNTQUKJRHANCNFSM4JOKCTQQ.

--

Martin Jung Dipartimento di Fisica Università degli Studi di Torino

DavidMStraub commented 5 years ago

@dvandyk @talismanbrandi

Look at these plots: https://gist.github.com/DavidMStraub/daf2cf5bedcd7b3637e235bf19e328ea

These looks pretty darn Gaussian.

No way the huge discrepancy I'm seeing has to do with that. It's probably some other stupid bug.

DavidMStraub commented 5 years ago

The relation should be 1/2 xi''(1) = c.

Then it's correct :wink: LHS is c, RHS is your data (xi'')

peterstangl commented 5 years ago

Comparing https://gist.github.com/DavidMStraub/daf2cf5bedcd7b3637e235bf19e328ea to https://gist.github.com/DavidMStraub/778f8acc2261733b4aeeb21531e85195 and table II in https://arxiv.org/pdf/1908.09398.pdf, it seems to me that the xi' -> rho and xi'' -> c transformations have been applied twice in https://gist.github.com/DavidMStraub/778f8acc2261733b4aeeb21531e85195, leading to a too small value of c_xi by a factor of 2 and a different sign for rho.

peterstangl commented 5 years ago

However, if rho = - xi', rho should be -1 according to table II in https://arxiv.org/pdf/1908.09398.pdf. Why is the parameter called CLN rho2_xi? is rho2 the square of rho?

peterstangl commented 5 years ago

Sorry, no, if rho = - xi', rho should be roughly +1 (I looked at f(1) instead of f'(1)). So all the values in https://gist.github.com/DavidMStraub/daf2cf5bedcd7b3637e235bf19e328ea seem to be fine while CLN rho2_xi and CLN c_xi in https://gist.github.com/DavidMStraub/778f8acc2261733b4aeeb21531e85195 seem to be wrong.

DavidMStraub commented 5 years ago

And there it is, the stupid bug! :rocket: Thanks @peterstangl :1st_place_medal:

(BTW the parameter is called rho^2 and it should be rho^2=-xi' IIRC)

peterstangl commented 5 years ago 
data = samples[:,:23]
data[:, 0] = -data[:, 0]  # xi' -> rho
data[:, 1] = 1/2 * data[:, 1]  # xi'' -> c

This is actually a bit dangerous. I think in interactive sessions one should always copy the data before applying such transformations:

data = samples[:,:23].copy()
data[:, 0] = -data[:, 0]  # xi' -> rho
data[:, 1] = 1/2 * data[:, 1]  # xi'' -> c

Just as a general comment on how to avoid these kinds of bugs :wink:

dvandyk commented 5 years ago

Interesting bug!

DavidMStraub commented 5 years ago

@peterstangl indeed that's the pit I fell into.

I confirm that this solves the issue.

Let's keep this open until we've added a unit test to prevent such things in the future (we could for instance check whether the sqrt of experiment/(theory/Vcb^2) is between 0.035 and 0.045 or so).