Qiskit Runtime primitive options difficult to find

jyu00 commented 2 months ago

URL to the relevant documentation

No response

Select all that apply

[ ] typo
[ ] code bug
[ ] out-of-date content
[ ] broken link
[X] other

Describe the fix.

At the Partner Forum I heard again that Qiskit Runtime primitive options are difficult to find in the docs. We attempted to fix this by adding a table in #1124, but perhaps that wasn't enough. This issue is meant to brainstorm on how we can improve on the experience.

javabster commented 2 months ago

one option is to link to the table in the API reference more: https://docs.quantum.ibm.com/api/qiskit-ibm-runtime/options#classes

jyu00 commented 1 month ago

[x] Based on some other feedback, I suggest we rename the Advanced runtime options page to Overview. This page will have these sections:
Introduction
[ ] - Explains the options tiered structure
[x] Explains the client default (Unset) and where to find server defaults
[x] Explains option precedence rules (related issue)
[ ] - Options categories for v1/v2 sampler/estimator. Something similar to the current table, but maybe as clickable tabs and no code examples, to make it more readable.
[ ] - Setting options for V2 primitives
[ ] - Setting options for V1 primitives
[x] - Commonly used options
[x] - Turning off error suppression and mitigation (related issue).
[x] I'd also suggest we rename Configure runtime compilation to Configure error suppression and focus on dynamical decoupling and twirling. Now that we don't transpile anymore in the primitives, "compilation" becomes less interesting compare to "error suppression".

One last comment - having both V1 and V2 information on the same page makes it really cluttered. Is it possible to have separate pages for V1 and V2 primitives? That's also make clean up easier when it's time to drop V1.

@javabster let me know what you think 🙂

beckykd commented 1 month ago

One last comment - having both V1 and V2 information on the same page makes it really cluttered. Is it possible to have separate pages for V1 and V2 primitives? That's also make clean up easier when it's time to drop V1.

I'm not sure about this. Of course it would be possible, but we don't split out the pages anywhere else; so would we want to then split them out everywhere? Also, having two sets of information can easily lead to them getting out of sync. And might people be using the V1/V2 tabs to better understand the changes? We of course have the migration guide, so this shouldn't be a problem, but it might be helpful.

That being said, I'm open to discussion.

beckykd commented 1 month ago

Something similar to the current table, but maybe as clickable tabs and no code examples, to make it more readable.

Totally on board with making this better. What would the tabs be? V1 Estimator, V1 Sampler, etc? I think that would be nice / doable.

[ ] I really like the code examples, as a non-coder. What if we re-formatted it like this:

Category	Description
Resilience	Advanced options for configuring error mitigation methods such as measurement error mitigation, ZNE, and PEC. Estimator only.
	estimator.options.resilience.measure_mitigation = True
Dynamical decoupling	Options for dynamical decoupling.
	estimator.options.dynamical_decoupling.enable = True

javabster commented 1 month ago

Jessie's ideas sound good to me, but I agree with Becky I think we should keep v1 and v2 together.

Also worth considering as part of this discussion is @kevinsung is working on an "error mitigation techniques" page that will also cover some of the runtime options, but it's more focused on explaining the error mitigation concepts.

[ ] we should just make sure that these pages flow nicely together and don't duplicate too much

kevin's PR: https://github.com/Qiskit/documentation/pull/1482

jyu00 commented 1 month ago

Something similar to the current table, but maybe as clickable tabs and no code examples, to make it more readable.

Totally on board with making this better. What would the tabs be? V1 Estimator, V1 Sampler, etc? I think that would be nice / doable.

Yeah I was thinking one tab for each V1/V2 sampler/estimator combination.

I really like the code examples, as a non-coder. What if we re-formatted it like this:

Sure. The only reason I suggested the removal is because the code example makes 1 column really wide and the others really narrow. If we can solve this without removing code example that's even better.

Also worth considering as part of this discussion is @kevinsung is working on an "error mitigation techniques" page

That's really good to know! There was some feedback from Partner Forum that researchers tend to want to know more about the underlying concepts, so this would be quite helpful.

jyu00 commented 1 month ago

Another suggestion. I saw a table with all the options and their defaults in a challenge notebook. Today to find the default value of a ZNE extrapolator, for example, I'd need to go into the API reference, click on EstimatorOptions, then ResilienceOptions, then ZneOptions, and search for extrapolator. And this is after already knowing where to look. A table would be super handy.

Having said that, the down side of a table is of course we'd have to keep it up to date.

Qiskit / documentation