search

microsoft / Qcodes

Modular data acquisition framework
http://microsoft.github.io/Qcodes/
MIT License
341 stars 315 forks source link

Hierarchy #5

Closed guenp closed 8 years ago

guenp commented 9 years ago

This is what I'd suggest for the object hierarchy, based on previous discussions in #2. See also #4 for discussions on the Monitor.

sweep_monitor

AdriaanRol commented 9 years ago

I'll comment here as the V0 PR has already been merged and it is a bit unclear to me what discussion belongs where at the moment altough this would probably fit better in the (now closed) v0 thread as my comments are mostly on the general concept.

I think most of the points I will try to make have already been raised in some way or another by other people but they relate to the following points.

I just reread my post/comment and noticed that there is a lot of worries in there. I am sure that a lot of thought has been put into the design choices but we are a bit in the dark on what's happening in Copenhagen here in Delft.

Centrality of "sweep" and the measurement paradigm

I am a bit worried about how central the concept of a sweep is in the current measurement paradigm. This is indeed the most natural way to think about data acquisition when you are measuring things like stability diagrams but becomes harder when you are doing something like an AllXY pulse sequence or in a later stage things like surface code based quantum error correction.

Although it is possible to force these types of measurements into the "sweep" paradigm, (in fact we are doing so), there are two problems with this strict approach, the first relates to "sweeps" involving pulse sequences and the second relates to detection of quantities that are not instrument parameters.

To give an example for a problem with pulsed measurements, if we measure an AllXY sequence, we acquire data with a data acquisition card and we return data average data of ~1000s of shots each time. This is an array where the index corresponds to the element of the "sweep" that was acquired, however keeping track of the timestamps and other parameters during for each element of that sweep is not practically feasible. The second problem with seeing pulse sequences as sweeps is the question of what parameter is being sweeped, already in the relatively simple example of the AllXY sequence it becomes hard to answer this clearly.

The second problem I foresee is that with non-parameter quantities or derived quantities that are being measured. Suppose we want to 'sweep' a dac voltage while measuring T1, measuring T1 involves several smaller measurements (find the resonator, find the qubit, do a rough calibration for the pulse amplitude, perform a T1 experiment). Besides the requirement to "nest" experiments in some way the other issue that pops up here is that there is no instrument that has T1 as it's parameter and as far as I understand does not fit into the paradigm yet. Ofcourse it is possible to force T1 in as a parameter by creating an instrument specifically for that but I think this will only promote dirty hacks.

The proposed implementation of adding another argument to the sweep that sets the pulses to sweep is just not practical for us as it overly simplifies the complexity of doing pulsed measurements. 

measurement.sweep(B[-6:6:0.1],60).apply_pulse_scheme('my_pulse.dat', n=10000).monitor(t=600) #for each B, apply my_pulse 10.000x and monitor the system for 10 minutes

Defining measurements using multiple lines of code, as @guenp suggested, makes it both more readable and allows for more general types of measurements. The fact that this necessitates a run statement is not a downside to me.

I feel we have gone through a very similar development as what is being discussed now with respect to the measurement class/object. I would really like to share some of the lessons we have learned to make sure we get the best possible QCodes that is applicable to general physics experiments.

Modularity of the code

I am a bit worried about how modular it will be as it currently seems there is no clear distinction between data storage, the monitor and the instruments and the measurement object. It is not clear if I can just use the parts of QCodes I like and ignore what I don't at this moment but this will most likely work out just fine.

The instrument parameter class

One of the main complaints with QTLab was all the constraints that where put on the instrument parameters, such as the individual get/set functions and the data-type constraints. I don't know how the parameters will look but I am a bit worried there.

Practicality of data saving

Although the perpetual monitoring of data seems very nice in theory I think the usability of such a construct will be decisive in determining the succes of QCodes. If it is not easy to extract a subset of the data (e.g. of 1 experiment run) or separate datasets for easy sending it will be very hard to run analysis or share data with others, Then there is also the problem of multiple setups producing data and that being accessible and browsable in a nice way. Depending on how this is done I don't know if it is possible to teach an incoming master student how to search in a database on top of all the other things he has to learn.

akhmerov commented 9 years ago

@AdriaanRol

I am a bit worried about how central the concept of a sweep is in the current measurement paradigm.

Practicality of data saving... If it is not easy to extract a subset of the data...

These were parts of my motivation in #4.

alexcjohnson commented 9 years ago

@AdriaanRol Thanks for all these comments. Re: Copenhagen vs. Delft - don't worry, nobody here really knew what I've been up to until I posted the big PR #2 :see_no_evil: . I really wanted to have an end-to-end example for us to talk about so had been working in the dark for a while, but now that this is out I should be able to make PRs much more quickly to iterate this to something we're all excited about. And given that this is nobody's focus here except me, nothing is really going to happen offline here until we get to a point that people start using the code.

With that said, your first three points all tie in together in my mind. I've worked pretty hard to keep the couplings as light as possible, so it should be pretty easy to swap out whichever pieces don't fit your needs. This also relates to the discussion in #6 (syntax).

I'm not really sure how this works in QTLab, but here an object doesn't need to be a Parameter or part of an Instrument in order to be swept or measured in a sweep (or is it a Loop?). You can sweep anything that has a set method, and you can use slice notation if you just point its __getitem__ method at SweepValues. You can measure anything that has a get method and a name attribute. Instrument objects just serve to efficiently generate these parameters and bind them to communication channels (and maybe the monitor will use them to figure out what to measure by default), but you can use your own objects as you like.

So if I were to implement your T1 example, I'd make an object with a get method that executes the series of measurements you describe. Is that hacky? Feels fairly clean to me, you could even give it a constructor that lets you specify different control knobs and measurement channels for neighboring qubits.

I'm not familiar with the details of AllXY but I guess what you're saying is effectively the data is acquired in 1D series rather than point-by-point, where in this case each 1D series is really the average of many events. Good point, that will happen a lot! see https://github.com/qdev-dk/Qcodes/issues/6#issuecomment-149065012

Re: Practicality of data saving

I'm not sure if it's clear, but I see a sharp separation between experiment data and monitor data. We haven't talked at all here about the organization of either one, as the monitor isn't written yet and the experiment data I've so far just provided one example storage format (MergedCSVStorage) but punted on organization for now - it just asks for a disk location.

But what that means is the experiment data is all going to be stored in some simple format, where the class that saved it can also read it back in for later analysis. I had some other thoughts about how to make the experiment data easier to pull back in later.

Pulling up old monitor data - I expect mostly this is going to be for debugging (why did last night's data go screwy?) or reproduction (what were the gate voltages when I took that data?). You're right, we don't want a database for this, that people will need to learn SQL to query or something. I plan to just make a nice text format with well-organized file/folder names. Then most of the time people will just open the log file, scroll to the appropriate time, and look at it. But of course there will be times you want to plot data from the monitor - it should be fairly easy to write reader scripts for this, which won't take users long to learn. Sound reasonable?

guenp commented 9 years ago

One of the main complaints with QTLab was all the constraints that where put on the instrument parameters, such as the individual get/set functions and the data-type constraints. I don't know how the parameters will look but I am a bit worried there.

@alexcjohnson From what I understood from discussions with Delft, is that QTLab's instrument parameter structure is too rigid for advanced users (which most of us are by now). For e.g. parameter bla, the instrument driver would require you to create a do_get_bla and do_set_bla methods of the Instrument subclass, and 'normal' attributes wouldn't be saved in the log (snapshot) and/or data file. A request from the LK team (@damazter) was to add some kind of flag as to indicate which attributes to log or not. But I think Qcodes' MeasurementSet and Monitor classes could already cover this point, as here you just manually add parameters you want to log. However, this should thus also work for class attributes.

Furthermore, from the LdC team (@AdriaanRol) I got the same impression that they felt too restricted by the Instrument class, especially when creating complicated virtual instruments (such as qubit). The problems were mostly related to logging and data saving - QTLab only saves parameters that have a .do_get function. This forced them to create .do_get functions for each parameter they want to log, which simply return class attributes... This is kind of redundant and adds an unnecessary amount of lines of code...

Basically, we need a flexible way of saving and/or logging any parameter, whether it is a 'real' instrument parameter or a simple class attribute. One way I thought of solving this is with python3 properties :) this makes getting an attribute or an instrument parameter essentially the same thing! I do agree it might be a bit tricky to implement, but I solved this in athena by doing this (maybe a lil hacky :angel:):

attributes = inspect.getmembers(self, lambda a:not(inspect.isroutine(a)))
self.params = {a[0]:a[1] for a in attributes if not(a[0].startswith('__') and a[0].endswith('__'))}
alexcjohnson commented 8 years ago

Pretty sure most everything in here has been handled by the (not so new anymore) new syntax #7 but if there's anything left open feel free to make a new issue