Organize experiments according to QCVV definition

[eliarbel]

What is the expected behavior?

Not sure there is really a standard, text-book definition for QCVV, but it seems that the following definition is common:

• Characterization - estimating model parameters via experiments
• Verification - certifying the result of some control action (e.g. reaching a desired state)
• Validation - demonstrating quantum information processing capabilities

If we agree this is a good definition, then maybe we should organize the package experiments more in light of the QCVV categorization (we're close to it anyway), something like this:

Characterization: T1,T2Ramsey, EFSpectroscopy, QubitSpectroscopy, StandardRB, InterleavedRB

Verification: StateTomography, ProcessTomography

Validation: QuantumVolume

Calibration: Rabi, EFRabi, DragCal, FineAmplitude, FineSXAmplitude, FineXAmplitude

The suggested change is for the packaging and documentation structure, not for the repo folder structure

[ShellyGarion]

Perhaps you would like to consider the following classification according to the QCVV pyramid, which appear in many of IBM's slides:

[eliarbel]

You mean practically to split to "device", "subsystem" and "Holistic" categories ? That's an interesting view of qiskit-experiments, but personally I like the "standard" QCVV definition better. I think it's informative and probably more common. Plus currently the package is almost fully aligned with the "standard" QCVV view, so we can avoid a revolution in the way the exps are organized; just some adjustments are needed to get there.

[ShellyGarion]

I think that the division into categories is quite similar in both cases, namely: characterization <--> device verification <--> subsystem validation <--> holistic
I am not sure which terminology is more useful. Anyway, perhaps in the documentation one would like to add the QCVV pyramid?

[ShellyGarion]

Another comment: I think that StandardRB and InterleavedRB should be in verification and not in characterization. You can calculate a gate fidelity both using QPT and using StandardRB / InterleavedRB, so I think that RB methods should be in the same category as tomography methods.

[eliarbel]

I think that the division into categories is quite similar in both cases, namely:

I agree, which is why I prefer that we stick with the more commonly used one. Unless we would like to promote the pyramid usage as a standard categorization.

Another comment: I think that StandardRB and InterleavedRB should be in verification and not in characterization.

It's a really a matter of use-case. I think RB is more for characterizing gates, whereas tomography is for state/process verification. Sure you can use tomo to charecterize a single gate if that what you're circuit contains, but I would not use that argument to put RB and tomo together. Anyway, I'd be happy to hear other opinions here as well

[yaelbh]

@eliarbel @ShellyGarion Is this issue still relevant?

[eliarbel]

I think it's still relevant. @coruscating let's bring this up for discussion this week if possible