Exporter to WebUI

[jenshnielsen]

example data

{"type": "linear",
 "x": {"data": [0.0, 0.1, 0.2, 0.30000000000000004, 0.4, 0.5, 0.6000000000000001, 0.7000000000000001, 0.8, 0.9, 1.0], 
       "full_name": "dac_ch1", 
       "is_setpoint": true, 
       "name": "ch1", 
       "unit": "V"},
 "y": {"data": [1.00026278, 1.00026269, 1.00026319, 1.00026305, 1.00026248, 1.00026261, 1.00026245, 1.00026251, 1.00026226, 1.00026274, 1.00026266],
       "full_name": "keysightdmm_1_volt", 
       "is_setpoint": false, 
       "name": "volt", 
       "unit": "V"}
}

[jenshnielsen]

{"type": "heatmap",
 "x": {"data": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0, 10.0],
       "full_name": "dac_ch2",
       "is_setpoint": true,
       "name": "ch2",
       "unit": "V"},
 "y": {"data": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0, 10.0],
       "full_name": "dac_ch1",
       "is_setpoint": true,
       "name": "ch1",
       "unit": "V"},
 "z": {"data": [[48.0, 82.0, 99.0, 73.0, 14.0, 82.0, 13.0, 4.0, 10.0, 35.0, 33.0], [56.0, 58.0, 54.0, 81.0, 39.0, 10.0, 60.0, 97.0, 18.0, 73.0, 39.0], [78.0, 87.0, 72.0, 24.0, 55.0, 54.0, 15.0, 58.0, 83.0, 18.0, 23.0], [17.0, 52.0, 38.0, 17.0, 33.0, 1.0, 53.0, 18.0, 18.0, 73.0, 34.0], [20.0, 79.0, 68.0, 41.0, 71.0, 23.0, 8.0, 5.0, 17.0, 70.0, 63.0], [98.0, 82.0, 87.0, 64.0, 14.0, 35.0, 63.0, 7.0, 20.0, 13.0, 94.0], [52.0, 59.0, 89.0, 2.0, 85.0, 19.0, 36.0, 43.0, 67.0, 4.0, 34.0], [92.0, 90.0, 64.0, 76.0, 95.0, 6.0, 37.0, 31.0, 47.0, 67.0, 90.0], [44.0, 33.0, 46.0, 82.0, 75.0, 51.0, 72.0, 75.0, 87.0, 16.0, 29.0], [87.0, 83.0, 42.0, 52.0, 27.0, 22.0, 82.0, 66.0, 100.0, 100.0, 3.0], [2.0, 99.0, 21.0, 86.0, 17.0, 76.0, 25.0, 81.0, 22.0, 25.0, 31.0]],
       "full_name": "dmm_voltage",
       "is_setpoint": false,
       "name": "voltage",
       "unit": "V"}}

[sohailc]

@jenshnielsen Where can I find the branch in which you are addressing this issue?

[jenshnielsen]

There is no branch at the moment. The relevant export functions lives on the dataset branch. Before Christmas I got as far as doing a simple example in the webui of exporting from the database. But all that code lives in the example. Currently I cannot launch the webui because of the issue that we discussed yesterday that is waiting for a webui fix. The following will do a liveplot in the webui of the data from an experiment

import logging
import copy
logging.basicConfig(level="INFO")
from qcodes import Publisher
import numpy as np
import json

from qcodes.dataset.data_set import new_data_set, ParamSpec, DB, hash_from_parts
from qcodes.dataset.json_exporter import json_template_heatmap, json_template_linear
from qcodes.dataset.json_exporter import export_data_as_json_heatmap, export_data_as_json_linear
from qcodes.dataset.sqlite_base import new_experiment, connect, length

dataSet_hm = new_data_set("test", specs=[ParamSpec("x", "number"),
                                         ParamSpec("y", "number"),
                                         ParamSpec("z", "number")])

mystate = {}
xlen = 5
ylen = 10
mystate['json'] = json_template_heatmap.copy()
mystate['data'] = {}
mystate['data']['xlen'] = xlen
mystate['data']['ylen'] = ylen
mystate['data']['x'] = np.zeros((xlen*ylen), dtype=np.object)
mystate['data']['x'][:] = None
mystate['data']['y'] = np.zeros((xlen*ylen), dtype=np.object)
mystate['data']['y'][:] = None
mystate['data']['z'] = np.zeros((xlen*ylen), dtype=np.object)
mystate['data']['z'][:] = None
mystate['data']['location'] = 0

sub_id = dataSet_hm.subscribe(export_data_as_json_heatmap, min_wait=0, min_count=20,
                              state=mystate, callback_kwargs={'location': os.path.join('output', id, 'dac_ch1_set_dac_ch2_set.json')})

for x in range(xlen):
    for y in range(ylen):
        z = x+y
        dataSet_hm.add_result({"x":x, "y":y, 'z':z})
dataSet_hm.mark_complete()

[sohailc]

First a question: where does "id" come from in the line

os.path.join('output', id, 'dac_ch1_set_dac_ch2_set.json')

?

An observation: you can simply write:

mystate['data']['x'] = np.empty_like(xlen*ylen*[None])

[sohailc]

Also, how does the webui know where to look for the JSON file?

[jenshnielsen]

When the lab station runs the generated python script it appends the following in front

    "import sys, os",
    "import qcodes as qc",
    "sys.path.append(r'{app_root}')",
    "from data.formatters.json_formatter import JSONFormat",
    "id = sys.argv[1]",
    "loc_provider = qc.data.location.FormatLocation(fmt=os.path.join('output', id))",
    "qc.data.data_set.DataSet.location_provider = loc_provider",
    "qc.data.data_set.DataSet.default_formatter = JSONFormat()"

defined in default.json so the id comes from the lab station