search

Qiskit / qiskit

Qiskit is an open-source SDK for working with quantum computers at the level of extended quantum circuits, operators, and primitives.
https://www.ibm.com/quantum/qiskit
Apache License 2.0
4.99k stars 2.32k forks source link

Qiskit docstrings are not validated #12582

Open nonhermitian opened 2 months ago

nonhermitian commented 2 months ago

URL to the relevant documentation

No response

Select all that apply

Describe the fix.

None of the docstrings in the Qiskit docs appear to be executed for validity. This leads to many broken code snippets, out of date / non-working code, etc. I would expect all the examples in the Qiskit docs to be fully functional for the Qiskit version they correspond to.

javabster commented 2 months ago

As far as I'm aware the way to do this validation is to convert the docstrings to use doctest, so this would need to be done in the qiskit repo where the docstrings are written. I'll transfer this issue there

1ucian0 commented 2 months ago

Hi @nonhermitian, It is relative expensive to do it for all of them. Which ones are critical and wrong? We can start with those.

nonhermitian commented 2 months ago

It is expensive, but that is really not an issue at the end of the day. Namely we have RC's now, and the expensive calls to eval doc strings can probably be configured to run only on certain PRs or releases so that it need not be run all the time. Doing so would mean you do not have to ask me what ones are wrong after the fact.

jakelishman commented 2 months ago

Paul: that is true, but setting up docstring testing against examples which (I'm guessing?) often involve calls to external services or IBM-specific simulators that can change from underneath Qiskit's control will be a significant amount of work right now to do across the library. It's very much not as simple as setting a "run docstring tests" flag.

In the interests of prioritisation, right now, you could help us out by telling us which examples you know of that are broken. We may or may not have the resources to configure docstring testing for those examples and to rewrite them to be doctest-safe, but we can at least fix the ones you've seen if you tell us about them.

nonhermitian commented 2 months ago

Can start here I guess: https://github.com/Qiskit/qiskit/issues/12583 But I think there needs to be a plan going forward on how to guarantee the docs return valid content.

nonhermitian commented 2 months ago

This would be nice to sort out as well: https://github.com/Qiskit/qiskit/issues/12589

nonhermitian commented 2 months ago

TwoLocal doc strings attempt to add two circuits which fails: https://docs.quantum.ibm.com/api/qiskit/qiskit.circuit.library.TwoLocal