Accessing complex types via control server

[MikeHart85]

This is from a question asked by Tom Willemsen.

If a device is defined like this:

class MyDevice(StateMachineDevice):
    def initialize_data(self):
        self.channels = {1 : Channel(), 2 : Channel(), 3 : Channel()}
        ...

Where Channel is defined like this:

class Channel(object):
    def __init__(self):
        self.waveform_type = 0
        self.step_time = 0
        ...

Would it be possible to access via control server using this sort of syntax (or similar):

$ lewis-control device channels[1].waveform_type 1

For now I suggested creating set_channel_param(self, index, param, value) and get_channel_param(self, index, param) methods on the MyDevice object. But it would be nicer if we could avoid the need for creating such boilerplate methods.

There are two difficulties with implementing something like this:

• Syntax / shell interaction ([1] might mean something special in some shells)
• Trying to json-rpc Channel objects throws a "not serializable" exception

The method workaround works fine, so this issue is mainly to see if we can come up with ideas on how to improve this.

[MichaelWedel]

One possibility that came to my mind would be to expose objects via the control server recursively.

But while this would be a nice option for "non trivial members" (for example a member of an exposed object that is of some non-standard data type), it quickly gets complicated when taking into account collections of such things, as demonstrated in the example.

If we go down this road too far, we end up implementing an entire remote object broker thing and there are most likely better solutions around for that.

Another option would be considering this: https://jsonpickle.github.io/, but I'm not entirely sure yet how much effort that would require - and it would not solve the problems you pointed out regarding the command line usage. For such complex queries it might be warranted to write a Python script, possibly with its own argparser?

[MikeHart85]

Played around with jsonpickle a bit... it looks really good, nice find!

Seems to have no trouble encoding and decoding even complex types that contain other complex types (Like SimulatedChopper with its self._bearings = SimulatedBearings()).

Only requirement seems to be that the type is importable (but not necessarily imported yet) under the same full name (like lewis.devices.chopper.devices.device.SimulatedChopper). If it's not importable, decoding produces a dict object instead.

I don't think we need to worry about supporting weird things like this on the command line too much, the most important part would be for it to work from the Control Client Python API (without having to define special getters/setters on the device).

[chris-d-gregory]

We have a related problem: values that are returned from the interface as bytes objects aren't accessible in lewis-control since they are not JSON-serializable either. The change to jsonpickle suggested above should also fix our issue. If no one is working on it now I'm happy to take a look.

[mattclarke]

Hi. @chris-d-gregory, if you have time to take a look then that would be great :D

[chris-d-gregory]

Hi @mattclarke - I'm looking at this now, and have some progress. I've swapped out jsonrpc for what is essentially a copy of that library but that uses pickle for serialisation. That has at least allowed me to read device properties encoded as bytes from lewis-control. I need to clean it up and do further tests (and write tests and docs), but I'd like to issue a draft PR to get some feedback. There is no feature branch for this in the repo - would you like to create one since I don't have the permissions? Or should I target an existing branch?

[mattclarke]

Hi @chris-d-gregory. I've sent an invite to add you as an outside collaborator. Provided you are happy with that responsibility ;) - it should allow you to create a branch etc. Shout if it doesn't!

[chris-d-gregory]

Excellent, thanks. Just successfully created the branch. I'll push what I have so far soon as a test, although it's not ready for review yet.