polarization fidelity is not a valid comparator

[nonhermitian]

The QED-C benchmarks, and paper, use the polarization fidelity as the comparison metric amongst applications and differing numbers of qubits. This is given by Eq.(2) of the paper (https://arxiv.org/abs/2110.03137). In the plots this fidelity is reported on the interval [0,1], e.g

. However, the polarization fidelity is not defined over the interval [0,1]. Instead, the lower bound is negative and is given by

-(1/2**N)/(1-1/2**N)

where N is the number of qubits. Therefore the range [0,1] is only valid in the large N limit. More importantly, this lower bound is qubit number specific. Therefore this fidelity cannot be used as a comparator across differing numbers of qubits, as done in the tests and paper.

The polarization fidelity should, for example, be shifted and rescaled so that the range [0,1] is valid across all numbers of qubits.

[nonhermitian]

Actually the above is valid save for the case where the ideal distribution is also the uniform distribution for which the denominator of Eq.(2) goes to zero. I am not sure how to solve that pathological case.

[rtvuser1]

Yes, that is one of the issues that the team wrestled with (the divide by zero case). We settled on polarization fidelity (as defined in the repository) as a good compromise that addresses the requirements of the current suite of benchmark programs.

On Sun, Feb 13, 2022 at 7:41 AM Paul Nation @.***> wrote:

Actually the above is valid save for the case where the ideal distribution is also the uniform distribution for which the denominator of Eq.(2) goes to zero. I am not sure how to solve that pathological case.

— Reply to this email directly, view it on GitHub https://github.com/SRI-International/QC-App-Oriented-Benchmarks/issues/47#issuecomment-1038202471, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADIPLWUOJKEONW2BWXHDGJDU27GJXANCNFSM5OJENK2Q . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you are subscribed to this thread.Message ID: <SRI-International/QC-App-Oriented-Benchmarks/issues/47/1038202471@ github.com>

[nonhermitian]

Actually I messed up the math above, maximizing the wrong thing. The worst case is given by an experiment that returns a single bit-string, but the actual answer is a uniform distribution over all other bit-strings. In this case one gets an overlap with the uniform distribution of 1-1/2**N so the lower bound is:

-(1-1/2**N)/(1-((1-1/2**N))

Which a quick check equals -2^{N}+1. Which is still qubit number dependent.

In short, the more important point is not really the zero division, but that your are equating things with fidelities over different ranges. And the fact that the lower bound is negative, but that is not anywhere in the plots.

[necaisej]

It should be specified in the paper that we are reporting all negative fidelities as 0, for the following reason; forgive me if this is repeating what you already know, but this is a representation of the thought process behind the implementation:

The choice of normalizing to some large-noise limit distribution, in this case the uniform distribution over bitstrings, is intended to clarify the actual ability of a quantum processor to execute its instructions (without tomography) and also its ability to achieve a task better than some easy classical task (random guessing). Polarization fidelity is negative when the Hellinger fidelity of the large-noise limit distribution with the exact distribution is higher than the Hellinger fidelity of the actual outputs with the exact distribution. So "less informative than noise" is conflated with "indistinguishable from noise." It's a fair criticism of the metric that these are not quite the same cases but in reporting whether your QPU can perform the task that an algorithm is designed to do, we are currently choosing to set them both as equal to 0.