Make sure that error mitigation techniques are tested for all SUPPORTED_PROGRAM_TYPES

[andreamari]

For example, at the time moment, zero-noise extrapolation is never tested with Braket circuits (see #767).

A good way to automatically test all (current and future) supported circuit types is to loop over mitiq._types.SUPPORTED_PROGRAM_TYPES when writing important tests. For example this was done here: https://github.com/unitaryfund/mitiq/blob/ff6d3e540d7f14430a3dfcf4f7d1f1a5bdc50109/mitiq/cdr/tests/test_cdr.py#L36

[nathanshammah]

Great. Maybe we could add something about it in the developers' guide?

[rmlarose]

This could use more detail / clarification.

• What error mitigation techniques are already tested with all support program types? and which ones need more tests?
• Is the issue to add tests or develop a framework to ensure they're always tested?
  • The latter could be an addition to the developers guide as noted above. But it would be better to have infrastructure than a note hidden somewhere.

[andreamari]

• What error mitigation techniques are already tested with all support program types? and which ones need more tests?

Initially only execute_with_cdr was tested with all SUPPORTED_PROGRAM_TYPES.

A recent PR #769 , added a similar test for execute_with_zne.

I just opened #779, which does the same for execute_with_pec. So, if #779 will be merged, tI think his issue can be closed.

• Is the issue to add tests or develop a framework to ensure they're always tested?

My proposal was to just add some tests with a loop over SUPPORTED_PROGRAM_TYPES. In this way, if in the future we add a new circuit type, such tests will be automatically up to date. I didn't have any advanced infrastructure in mind.

[rmlarose]

Thanks Andrea, sounds good.

I didn't have any advanced infrastructure in mind.

It might not be a bad idea to have a test like the following:

• Find every execute_with* function.
• For each one, raise an error if there is not a line that contains @pytest.mark.parametrize("circuit_type", SUPPORTED_PROGRAM_TYPES.keys()) in the corresponding module.

It could probably use a design document - e.g., we probably want to be able to ignore modules in development - but that's the basic idea.

Or we could just be more careful on reviews in the future. But I tend to like automating things.