More flexible Subsystems

[MatthieuDartiailh]

The idea comes from the discussion in the python-ivi thread. When building complex instruments having to rebuild the driver hierarchy of subsystem can be tedious and counter productive. The idea would be to mark some Features as being part of a subsystem (how ? perhaps using a context manager in the declaration) and then let the metaclass build the hierarchy itself. Of course we should then provides way to access the hook of the feature directly from the class for subclassing. I think this would make subsystem more user friendly (even if a bit more magical). This should all be possible but will make the metaclass handling a bit tougher.

[hgrecco]

The SCPI standard is hierarchical and is divided in subsystems. I think we need to think from the users point of view. How would we like the user to use this? The implementation will follow.

[MatthieuDartiailh]

Actually I tend to find SCPI a bit over-divided. For example the only driver I implemented in Eapii is a standard DC voltage/current source. To set the voltage, you must sure the instrument works as a voltage source and set SOURce:LEVel. As a user I find it more natural to have a voltage property (which actually checks the operation mode) at the root rather than access source.level. If we go with something so divided it might make sense to provide Alias (just like SCPI allow to use shorter commands).

[MatthieuDartiailh]

My idea here is to explore the possibility to use special markers (context manager, decorators, etc) to allow to modify or extend a subsystem directly from the main driver so that you do not need to first subclass the system then the driver and put everything back together. I think this is doable, but will require more metaclass magic and lead to automatic subclassing of some subsystems. I am not sure yet it is viable but it would address some of @alexforencich worries.

[alexforencich]

Another thing is that the SCPI 'hierarchy' components can be omitted in many cases. So you could just send voltage:level or even just level to set the voltage. There are some things I like about the way IVI does things. For power supplies, you get instance.outputs[k].voltage_level and instance.outputs[k].current_limit. Also, the strange thing about SCPI for power supplies is that the individual outputs are addressed with a separate command to select the output followed by commands to adjust that output, as opposed to using a unique prefix like channel1 or source1 or similar.

[MatthieuDartiailh]

That kind of oddities is why I think it important to be able to customize how channels send their call to the root driver (ie the one with access to the communication pipe). By the way, I like the idea of having an array like container for channels, I will add it on the wiki.

[alexforencich]

Heh, that is one thing that took me a while to figure out in python-ivi. If you haven't already, please take a look at the implementations of PropertyCollection, IndexedPropertyCollection, and IviContainer in python-ivi: https://github.com/python-ivi/python-ivi/blob/master/ivi/ivi.py#L140 . These are the objects that provide the separable hierarchical user-facing API for the otherwise flat implementation. Now, this may not be the absolute best way to do this, but that's what I came up with. With this setup, the list is iterable and it can be updated on-the-fly with a new set of indicies. It's also possible to index by name ('channel1', 'channel2', ..) or by index (0,1,2,...). This way the driver can, say, auto-detect the number of channels installed and populate the list accordingly. On the other hand, the way this is set up all of the members end up being identical. Which may or may not be a problem depending on the instrument.

[MatthieuDartiailh]

I do have the notion of channel, the only thing is that I do not expose the container but rather a method to get the channel object. This is nice but does not direct iteration (you can always queries the list of defined channels and iterate the getting method over it). I already took a look at those classes, they look nice but building up validation might not be easy furthermore they rely on a __getattribute__ override that I would prefer to avoid (Python is not lightning fast but I would prefer to make it even slower).

[alexforencich]

Yeah, as I said there may be a better way to implement it but I'm not sure what the best approach would be.

[MatthieuDartiailh]

I try to put in place my metaclass magic and let you know but I will need a bit of time to figure it out.

[alexforencich]

Ah yeah, metaclasses. That is one area of python that I have very little experience with.

[crazyfermions]

Is there yet a more detailed idea of providing a unified API for certain subclasses of instruments for lantz drivers? I do not think it needs to be as strict as the IVI specifications, but there are many good ideas in there. For example, it introduces these so called "repeated capabilities", which would be exactly what Matthieu called an array of channels.

[hgrecco]

@crazyfermions We do not have any concrete plan but how about if we start doing it. I would suggest that we create an issue or wiki for each type of Device/Subsystem and we start brainstorming the API. By the way another thing to discuss is when to use a property and when to use a method. This has been a continuous source of questions.

[MatthieuDartiailh]

Yes I do think it is a great idea to have that kind of brainstorm. The question of property vs method is indeed a tough one, for example I would be in favor of having method for things related to data acquisition (measure, fetch, etc). I have started to work on a possible implementation of the more flexible subsystems, once I have done some progress I will open a PR just so that people can give look at how it may work.

[MatthieuDartiailh]

Hi, So I got a chance to write a possible subsystems implementation. It tried to make few hypothesis on what we will decide for other part of the code. As this is still WIP and that there are still some Eapii concept hanging around I won't directly open a PR (you can always look at the code in my repo of lantz_core, mostly has_features.py), but I will describe what it can do so that we see whether or not it fits our needs. Please comment ! (@p3trus, @arsenovic feel free to join the discussion)

Let's start with some code that would be valid in that implementation :

from lantz_core.has_features import subsystem, set_feat, channel
from lantz_core.base_instrument import BaseInstrument
from lantz_core.features.feature import Feature

class GenericLockIn(BaseInstrument):
    """
    """
    #: Mode of acquisition of the lock-in
    acquisition_mode = Feature()

    #: Oscillator subsystem
    oscillator = subsystem()
    with oscillator as o:

        #: Amplitude of the oscillator in V.
        o.amplitude = Feature()

        #: Frequency of the oscillator in Hz
        o.frequency = Feature(getter=True)

        @o()
        def _get_frequency(self, feature):
            return feature

    channels = channel('list_channels')
    with channels as c:
        c.x = Feature()

    def list_channels(self):
        return 1, 2, 3

class SpecialisedLockIn(GenericLockIn):
    """
    """
    oscillator = subsystem()
    with oscillator as o:
        o.frequency = set_feat(secure_comm=2)

        @o()
        def _get_frequency(self, feature):
            return self

So let me explain what goes on here. The idea is to create a generic lock-in driver in which the parameters of the internal oscillator make a subsytem (and which has several acquisition channels, but that is just to show how channels might work). I assumed we were going to use some kind of descriptor similar to Lantz Feature or Eapii IProperty.

In GenericLockIn :

• first we create the acquisition_mode property (without any arguments as we have not discussed that yet) with a comment starting with #: which is recognized by sphinx as documentation which will be used to build the docstring of the descriptor.
• then we declare a subsytem named oscillator. The subsystem object will be used by the metaclass to create a custom subclass of SubSystem fitting our needs. Here we do not specify any base class but we could to get some previously declared behavior, actually we can specify several and take benefits of using mixins. Note that here also the comment will be used as docstring. Now we add some features to our system. As the name oscillator is a bit long we use a context manager to get a short alias. We can now add features to our subsystem and use Eapii customization hooks (if we keep them) to define a special behavior (note the decorator to attach the method to the subsystem, this allows to avoid naming conflicts, actually I don't think we need a decorator with args so we could remove the ()).
• finally we add a channel declaration, which takes as argument the name of the method used to list the available channels (should return a list of their ids). And we can proceed just like for subsystem for the features and methods. NB : We could of course go deeper in the nesting of subsystems and channels.

When you instantiate a generic lock-in, one instance of each subsystem is created and the channel attribute is given a ChannelContainer (channel.py) which behaves like an iterable and can be used to access to channel by id.

Now lets say you want to customize the frequency of the oscillator. You subclass the GenericLockIn, declare the subsystem oscillator and change the feature behavior either using the set_feat object which allow you to change any argument of the feature or by creating a new get method (or both). The metaclass takes care for you of subclassing the GenericLockInOscillator class (created by the metaclass when creating GenericLockIn) and can possibly add new mixin classes if you need to, and of customizing the features.

Old eapii ways to declare a SubSystem (Channel) by directly passing a class as class attribute are still supported but do not make sense anymore.

This implementation, it seems to me, solves a number of the issues related to the implementation of hierarchical drivers we mentioned in the labpy-discussion wiki, while still allowing to use descriptor and avoiding to override getattr or setattr. The drawback being a longer class creation time and hence importing but that need to happen only once in each application, and also a more complex codebase but nothing we can deal with (even if a strong testsuite will be imperative). Once things I like a lot also is that the hierarchy of the declaration reflects the hierarchy of the driver. I am open to suggestion remarks and comments, and I would prefer to have some feedback before cleaning the code a bit and writing a full testsuite (the example does work with the current implementation).

[alexforencich]

That actually looks quite clean. I like it. I presume list_channels be overridden in a subclass? And can the channel list be updated dynamically (e.g. after querying for installed cards)?

[hgrecco]

I agree with alex. It looks very clean and maintainable. +1 Subsystems +1 on the context manager.

There is another way to get the channels (and any other values which are only available after initialization). Lantz has the concept of Self, which we got from Traits. It is a Singleton object that references the current 'object'. But after implementing it and showing other people how to use it I think your approach is simpler and easier to maintain. So I will be happy to drop it.

Not so sure though about how we write the getters and setters. The way python does it normally with properties (i.e. with decorators) is nice and pretty established. In any case, this is another issue.

If you allow me to do some bikesheding, I like the word Driver more than Instruments. The instrument is the physical device but the program that we do drives the instruments. I think this distinction between instrument and driver is useful. It is also how National Instrument does it, python-ivi (correct me Alex if I am wrong) and others do it.

[alexforencich]

I'm wondering if it makes more sense to start with a clean repo than to try to graft all of this on to lantz. It seems like there are a number of architectural changes that need to be made to lantz already, not to mention all of the existing drivers will probably need to be rewritten.

And I definitely think using 'driver' instead of 'instrument' makes the most sense.

[MatthieuDartiailh]

I left the name Instrument because for now forgotten reasons I used it in Eapii. I can definitively change it. For the time being, when asking for available channels or iterating over them the list channel method is always called so the list is de-facto refreshed. If we add a cache (we probably should), we can easily add a refresh method on the container. As the ChannelContainer only cares about the name of the method to call you can definitively override list_channels in subclasses.

Setter and getter are actually another matter but I will answer nonetheless. In Eapii, I chose to drop the need to systematically write a getter or setter. Here is what I do. Each HasFeatures class implement a default_get/set_iproperty. When an IProperty has received a get/set argument it is considered gettable/settable, and when the corresponding operation is required the argument is passed to the default_get/set method. In this way for Visa system, you can have a default_get relying on query and the get argument can be simply the string to pass. In this way we avoid a bunch of duplicated code.

I agree with @alexforencich that it makes things messy to try to incorporate things in lantz right away. I wil open a bunch of generic issues on labpy-discussion about code organisation, subsystems (I will just link this issue), getter and setter and basically everything related to the core part. We can then start a new repo (and rename it later) just so that things does not get too messed up.

[arsenovic]

@MatthieuDartiailh , im having trouble finding your supporting code for this. can you point me to it?

[MatthieuDartiailh]

Oh yeah sorry about that it is in the merge-eapii branch .... Here is a link : https://github.com/MatthieuDartiailh/lantz_core/tree/merge_eapii.

[MatthieuDartiailh]

By the way I opened a bunch of new issues in labpy-discussion about how to move forward with this great project.