Review the data fields from provider, backends, and job objects

[jaygambetta]

Going through the tutorial I noticed many things not the same as the spec. I will list them but I want to start the thinking about what needs to be done to get the spec and fields to match.

Obvious ones

• [x] backend.status
• [x] backend.configuration
• [x] backend.properties
• [x] job.status
• [x] job.results
• [x] generally i want to see some way that the schema are passing.

backend.status

the structure returned does not match spec:

'name' should be 'backend_name'
'backend_version' is missing
'status_msg' is missing

backend.configuration

configuration structure returned does not match spec:

'name' should be 'backend_name'
'version' should be 'backend_version'
'basis_gates' should be a list of strings
'gates' is missing
'conditional' is missing
'chip_name' should be 'sample_name'
'open_pulse' is missing for atlantis
if 'open_pulse' is present then these also should be present:

                      "open_pulse","n_uchannels","hamiltonian","u_channel_lo",
                      "meas_levels","qubit_lo_range","meas_lo_range","dt","dtm","rep_times",
                      "meas_kernels","discriminators".

'allow_q_object' is not in spec but is allowed
'internal_id' is not in spec but is allowed
'deleted' is not in spec but is allowed

backend.properties

properties is empty on some. For melbourne:

'backend' should be 'backend_name'
'multi_qubit_gates' should be 'gates'
the gate structures in multi_qubit_gates does not match gate_prop in spec
'fridge_parameters' should be formatted as in ['general']['general_parameters']
'qubits' are not in nduv format. list[dict[dict]] should be list[list[dict]]

job.status

confirm spec changes match the code.

job.results

job.result() returns a Result object instead of a dictionary as specified in the spec.

The Result object is missing the properties: 'backend_version', 'qobj_id', and 'success'. It does, however, have Result.status.

The list of exp_result returned by Result.results() is an experiment result object instead of a dictionary. It is missing the properties: 'shots', 'success'.

fixes #1150

[jaygambetta]

in #1121 the status was fixed

[nonhermitian]

Simulator backends do not have n_qubits in their configuration. They are supposed to return -1.

[delapuente]

This is the first of a series of comments focused on listing the different data structures and interfaces inside Qiskit backends infrastructure. It is aimed at reflecting the real state of the current definitions and implementation in the several sources referring to it.

Symbol	Meaning
✓	Present
x	Absent
-	Does not exist

backend

Code: qiskit.backends.basebackend.BaseBackend.

Attribute	In the spec	In the code	In the schema	Action	Done
name	✓	✓	-
provider	✓	✓	-	Make it a method.	Done #1312
configuration	✓	✓	-
properties	x	✓	-	Make them optional for simulators according to (1)
defaults	✓(2)	x	-
schema	✓	x	-	Remove; see validate.
status	✓	✓	-
run	✓	✓	-
jobs	✓	x	-
retrieve_job	✓	x	-
calibration	x	x	-	Deprecated; remove	Done #1305
parameters	x	x	-	Deprecated; remove	Done #1305

(1) Introduced in #1401: properties refer to physical properties so simulators can or cannot return these. If None is returned, properties are assumed to be ideal.

(2) Only for OpenPulse

backend.validate

The backend run method accepts a QObject object to be executed. This object can come from untrusted origins. With "untrusted", we are not only referring to malicious sources but also to incorrect QObjects coming from flawed transpilation processes. A validation process is required to confirm that the running QObject is semantically correct.

backend.configuration

Code: qiskit.backends.models.backendconfiguration.BackendConfigurationSchema Schema: qiskit/schemas/backend_configuration_schema.json

Attribute(*)	In the spec	In the code	In the schemas	Action
backend_name	✓	✓	✓
backend_version	✓	✓
n_qubits	✓	✓	✓	Update the spec according to (1)
basis_gates	✓	✓	✓
gates	✓	✓	✓
local	✓	✓	✓
simulator	✓	✓	✓
conditional	✓	✓	✓
configurable	✓	✓	✓	Make it false by default
n_registers	✓	✓	✓
register_map	✓	✓	✓
open_pulse	✓	✓	✓(2)	Thinks about renaming
online_date	✓	✓	✓
display_name	✓	✓	✓
sample_name	✓	✓	✓	Clarify in the spec what a sample is.
description	✓	✓	✓
url	✓	x	x	Remove
tags	✓	✓	✓
memory (3)	x	✓	✓	Add to spec
max_shots (3)	x	✓	✓	Add to spec
max_experiments (4)	x	✓	✓	Add to spec
coupling_map(5)	x	x	✓	Make it required in schemas
credits_required(6)	x	✓	✓

(*) Validation introduced in #1277

(1) Introduced in #1388. This field no longer return -1. What's the new value for simulators?

(2) Indicates the configuration is for an OpenPulse backend or an OpenQASM one.

(3) Introduced in #1391

(4) Introduced in #1402

(5) coupling_map is in the schema but not required, nor explained in the specs nor validated in the code.

(6) Seems very specific and it is not in the specs. Should we remove it from the code and schemas?

backend.configuration.gate_config

Code: qiskit.backends.backendconfiguration.GateConfig Schema: qiskit/schemas/backend_configuration_schema.json

Attribute(*)	In the spec	In the code	In the schemas	Action
name	✓	✓	✓
parameters	✓	✓	✓
coupling_map	✓	✓	✓	Make it mandatory in the schemas and code
qasm_def	✓	✓	✓
conditional	✓	✓	✓
latency_map	✓	✓	✓
description	✓	✓	✓

(*) Field validation introduced in #1277

backend.properties

Code: qiskit.backends.models.backendproperties.BackendProperties Schema: qiskit/schemas/backend_properties_schema.json

Attribute(*)	In the spec	In the code	In the schemas	Action
backend_name	✓	✓	✓
backend_version	✓	✓	✓
last_update_date	✓	✓	✓
gates	✓	✓	✓
qubits	✓	✓	✓
general(1)	✓	✓	✓

(*) Field validation introduced in #1277

(1) Definition in the spec is ambiguous, it first says it is a list of nduv items but after, it says it is a list of general_parameters and refers to a non-existing parameters attribute.

backend.properties.nduv

Code: qiskit.backends.models.backendproperties.Nduv Schema: qiskit/schemas/backend_properties_schema.json

Attribute(*)	In the spec	In the code	In the schemas	Action
name	✓	✓	✓
date	✓	✓	✓
unit	✓	✓	✓
value	✓	✓	✓

(*) Field validation introduced in #1277

backend.properties.gate

Code: qiskit.backends.models.backendproperties.Gate Schema: qiskit/schemas/backend_properties_schema.json

Attribute(*)	In the spec	In the code	In the schemas	Action	Done
qubits	✓	✓	✓	Must be an integer	Done (#1427)
gate	✓	✓	✓
parameters	✓	✓	✓

(*) Field validation introduced in #1277

backend.status

Code: qiskit.backends.models.backendstatus.BackendStatus Schema: qiskit/schemas/backend_status_schema.json

Attribute(*)	In the spec	In the code	In the schemas	Action
backend_name	✓	✓	✓
backend_version	✓	✓	✓
operational	✓(1)	✓	✓
pending_jobs	✓	✓	✓
status_msg	✓(1)	✓	✓

(*) Field validation introduced in #1277

(1) In the spec, it says only operational and status_msg are required but in the schemas and the code, all of them are mandatory.

[delapuente]

provider

Code: qiskit.backends.baseprovider.BaseProvider.

Attribute	In the spec	In the code	In the schema	Action	Done
backends	✓	✓	-
get_backend	✓	✓	-
available_backends	x	x	-	Deprecated; remove	Done #1311

[delapuente]

job

Code: qiskit.backends.basejob.BaseJob.

Attribute	In the spec	In the code	In the schema	Action
status	✓	✓	-
job_id	✓	✓	-
backend	✓	✓	-
cancel	✓	✓	-
result	✓	✓	-
submit	x	✓	-

job.status

Code: qiskit.backends.jobstatus.JobStatus

Value	In the spec	In the code	In the schema	Action
ERROR	✓	✓	-
QUEUED	✓	✓	-
INITIALIZING	✓	✓	-
RUNNING	✓	✓	-
CANCELLED	✓	✓	-
DONE	✓	✓	-

job.result

Code: qiskit.result.result.Result. Schema: qiskit/schemas/result_schema.json.

Attribute(*)	In the spec	In the code	In the schema	Action
backend_name	✓	✓	✓
backend_version	✓	✓	✓
qobj_id	✓	✓	✓
job_id	✓	✓	✓
date	✓	✓	✓
header	✓	✓	✓
status	✓	✓	✓
success	✓	✓	✓
results	✓	✓	✓

(*) Validation for all properties added in #1277

job.result.resultobj

Code: qiskit.result.models.ExperimentResult. Schema: qiskit/schemas/result_schema.json.

Attribute(*)	In the spec	In the code	In the schema	Action
shots	✓	✓	✓
status	✓	✓	✓
success	✓	✓	✓
header	✓	✓	✓
seed	✓	✓	✓
meas_return	✓	✓	✓
data	✓	✓	✓

(*) Validation for all properties added in #1277

job.result.resultobj.data

Code: qiskit.result.models.ExperimentResultData Schema: qiskit/schemas/result_schema.json.

Attribute(*)	In the spec	In the code	In the schema	Action	Done
counts	✓	✓	✓
memory	✓	✓	✓
statevector(1)	✓	✓	✓	Go for option 1	Done (#1327)
unitary(1)	✓	✓	✓	Go for option 1	Done (#1327)
snapshots(2)	✓	✓	✓	Remove from spec/schema. Make it Aer specific.

(*) Validation for all properties added in #1277

(1) It seems there is some ambiguity regarding the type of unitary and statevector properties. It seems safe to assume that one statevector is an array of complex values while one unitary is an array of statevectors.

That said, the schemas define the properties unitary and statevector as collections (lists) of their respective types which do not match the spec. The spec defines unitary and statevector as single values of their proper types (the examples in the spec are wrong too).

The simulator is currently aligned with the JSON schemas and returns collection of the proper types.

Summarizing, there are two alternatives:

1. Define the fields unitary and statevector as single values of type "list of statevectors" and "list of complex" respectively.
2. ~Define the fields unitary and statevector as collections (lists) of values of type "list of statevectors" and "list of complex" respectively.~

(2) The snapshots field is loosely defined and it does not specify the types of snapshots nor the specific fields of each type.

[jaygambetta]

how is the easiest to comment on this.

[dcmckayibm]

@delapuente I would change a few of these tables to not have "in the schema" because you are listing functions for the objects and there are no schemas for those.

[diego-plan9]

Labelling as 0.7 preliminarly, to keep it under the radar.

[delapuente]

qobj.qobj_qasm

Code: qiskit.qobj._qobj.Qobj. Schema: qiskit/schemas/qobj_schema.json

Attribute	In the spec	In the code	In the schema	Action
qobj_id	x(*)	✓(1)	✓
schema_version	x	✓(2)	✓(3)
type	x	✓(2)	✓(3)
experiments	x	✓(1)	✓
config	x	✓(1)	✓
header	x	✓(1)	✓

(*) The OpenQASM flavour of the Qobj is not specified in the specification document. Should we introduce it and document there?

(1) These fields are currently required but not validated.

(2) Current implementation accepts extra parameters but they are not validated in any case.

(3) schema_version and type are mandatory in the schemas but not in the code.

qobj.qobj_qasm.config

Code: qiskit.qobj._qobj.QobjConfig. Schema: qiskit/schemas/qobj_schema.json

Attribute	In the spec	In the code	In the schema	Action
shots	x	✓(1)	✓
memory_slots	x	✓(2)	✓
max_credits(3)	x	✓(2)	✓
seed	x	✓(2)	✓

(1) These fields are currently required but not validated.

(2) Current implementation accepts extra parameters but they are not validated in any case.

(3) max_credits seems very specific to our backends.

qobj.qobj_qasm.header

Code: qiskit.qobj._qobj.QobjHeader. Schema: qiskit/schemas/qobj_schema.json

Attribute	In the spec	In the code	In the schema	Action
backend_name	x	✓(1)	✓
backend_version	x	✓(1)	✓

(1) Current implementation accepts extra parameters but they are not validated in any case.

qobj.qobj_qasm.qobjexp

Code: qiskit.qobj._qobj.QobjExperiment. Schema: qiskit/schemas/qobj_schema.json

Attribute	In the spec	In the code	In the schema	Action
instructions	x	✓(1)	✓
config	x	✓(2)	✓(3)
header	x	✓(2)	✓

(1) These fields are currently required but not validated.

(2) Current implementation accepts extra parameters but they are not validated in any case.

(3) The schema describes config as a way to replace the Qobj configuration per experiment but it does not specify the type of the attribute to be qobj.qobj_qasm.config

qobj.qobj_qasm.qobexp.header

Code: qiskit.qobj._qobj.QobjExperiment. Schema: qiskit/schemas/qobj_schema.json

Attribute	In the spec	In the code	In the schema	Action
name(1)	x	✓(2)	✓
n_qubits	x	✓(2)	✓
memory_slots	x	✓(2)	✓
clbit_labels	x	✓(2)	✓
qubit_labels	x	✓(2)	✓

(1) Experiment name is used in the Result API in methods (get_ran_qasm, data, get_counts, get_statevector, get_unitary, get_snapshots, get_snapshot and average_data) but it is not mandatory. The code treats this field as if it were compulsory. Should we do it mandatory?

(2) Current implementation accepts extra parameters but they are not validated in any case.

qobj.qobj_qasm.qobexp.openqasm_instructions (*)

Code: qiskit.qobj._qobj.QobjInstruction. Schema: qiskit/schemas/qobj_schema.json

Attribute	In the spec	In the code	In the schema	Action
name	✓	✓(1)	✓
memory	x	✓(2)	✓
params	✓	✓(2)	✓
qubits	✓	✓(2)(3)	✓
register	x	✓(2)	✓
texparams	✓	✓(2)	✓
label	x	✓(2)	✓
type	x	✓(2)	✓
conditional (4)	✓	x	x

(*) Instructions are oddly defined. In the spec, there are 7 types of instructions and one of them is "Gate" which define a generic gate with name and qubits as mandatory and optional params, texparams and conditional fields.

However, in the schemas are 14 types of instructions. They include the 7 mentioned before and also include all the standard library explicitly. Spec definitions and gate definitions are mixed together. Gates definitions are fine-tuned in these several types by providing restrictions on the number of parameters each specific gate accepts.

Some types in the spec are also merged with gate definitions. For instance, reset is defined along with id, h, s... I find this practice confusing since it does not allow differentiating between semantically different types of instructions (as is intended in the spec).

Some definitions, for which certain gate properties does not make sense include gate properties but constrained to be 0-length. For instance, the spec says snapshot has properties label and type but the schema also includes register, texparams, type and memory, all of them constrained to 0 items.

It is possible these constraints are added to avoid misunderstandings when parsing the instruction. If that's the case, we should document it.

Options here are:

1. Adding the standard library as part of the spec establishes a common model for a quantum computer although it prevents the user from providing custom definitions for the same gates and force 3rd-party backends to support this gate set.
2. The former could be worked-around by allowing 3rd party backends to fail when not recognizing some gates but that would imply they are not 100% specification-compliant.
3. Removing the gates from the schemas which would require an extension mechanism for circuits in place since the standard library would be provided by the IBMQ/IBM-simulators backends.

(1) These fields are currently required but not validated.

(2) Current implementation accepts extra parameters but they are not validated in any case.

(3) Mandatory in the spec but optional in the code.

(4) Its referred as register in the code and schemas, should be changed in the spec.

qobj.qobj_qasm.qobexp.bfunc

Schema: qiskit/schemas/qobj_schema.json

Attribute	In the spec	In the code	In the schema	Action
name	✓	x	✓(1)
mask	✓	x	✓
relation (2)	✓	x	✓
val (3)	✓	x	✓
register	x	x	✓
memory	x	✓	✓

(1) name is not mandatory in the schema.

(2) The list of valid relationships is missing. It should be open-ended.

(3) The representation is missing. Is it a signed or unsigned integer? A float? A double?

qobj.qobj_qasm.qobexp.copy

Schema: qiskit/schemas/qobj_schema.json

Attribute	In the spec	In the code	In the schema	Action
name	✓	x	✓
register_orig (1)	✓	x	✓
register_copy (1)	✓	x	✓

(1) source and target should be the common name for these and are shorter.

qobj.qobj_qasm.qobexp.barrier

Schema: qiskit/schemas/qobj_schema.json

Attribute	In the spec	In the code	In the schema	Action
name	✓	x	✓
register_orig (1)	✓	x	✓
register_copy (1)	✓	x	✓

(1) source and target should be the common name for these and are shorter.

[jaygambetta]

This looks great now

[delapuente]

I'm reopening this since there are several things not yet reviewed. I'm restoring the labels for tracking too.

[jaygambetta]

@delapuente i think we should close this and move the information about the qobj to the new issue and track it there as we finish the qobj

[ajavadia]

@delapuente can we get a summary of what is left to do here?

[ajavadia]

This appears to be done, or not receiving much attention in any case. If anyone is still looking into it, please reopen.