Error for unpick-able instruments

[damazter]

Similar to the situation described in QCoDeS/Qcodes_loop#13 A numerical simulation is written in instrument form and I try to run it with the following code

loop = qcodes.Loop(system.parameters["energy"][0:1:0.1], delay=0)
data = loop.run(background=True)

which gives the following error:

BrokenPipeError                           Traceback (most recent call last)
<ipython-input-8-996cbf03db06> in <module>()
      1 loop = qcodes.Loop(system.parameters["energy"][0:1:0.1], delay=0)
----> 2 data = loop.run(background=True)

d:\damaz\pycharm\qcodes\qcodes\loops.py in run(self, *args, **kwargs)
    171         '''
    172         default = Station.default.default_measurement
--> 173         return self.each(*default).run(*args, **kwargs)
    174 
    175 

d:\damaz\pycharm\qcodes\qcodes\loops.py in run(self, background, use_async, enqueue, quiet, data_manager, **kwargs)
    442             p.is_sweep = True
    443             p.signal_queue = self.signal_queue
--> 444             p.start()
    445             self.data_set.sync()
    446             self.data_set.mode = DataMode.PULL_FROM_SERVER

C:\Anaconda3\envs\kwant_env\lib\multiprocessing\process.py in start(self)
    103                'daemonic processes are not allowed to have children'
    104         _cleanup()
--> 105         self._popen = self._Popen(self)
    106         self._sentinel = self._popen.sentinel
    107         _children.add(self)

C:\Anaconda3\envs\kwant_env\lib\multiprocessing\context.py in _Popen(process_obj)
    210     @staticmethod
    211     def _Popen(process_obj):
--> 212         return _default_context.get_context().Process._Popen(process_obj)
    213 
    214 class DefaultContext(BaseContext):

C:\Anaconda3\envs\kwant_env\lib\multiprocessing\context.py in _Popen(process_obj)
    311         def _Popen(process_obj):
    312             from .popen_spawn_win32 import Popen
--> 313             return Popen(process_obj)
    314 
    315     class SpawnContext(BaseContext):

C:\Anaconda3\envs\kwant_env\lib\multiprocessing\popen_spawn_win32.py in __init__(self, process_obj)
     64             try:
     65                 reduction.dump(prep_data, to_child)
---> 66                 reduction.dump(process_obj, to_child)
     67             finally:
     68                 context.set_spawning_popen(None)

C:\Anaconda3\envs\kwant_env\lib\multiprocessing\reduction.py in dump(obj, file, protocol)
     57 def dump(obj, file, protocol=None):
     58     '''Replacement for pickle.dump() using ForkingPickler.'''
---> 59     ForkingPickler(file, protocol).dump(obj)
     60 
     61 #

BrokenPipeError: [Errno 32] Broken pipe

I feel that this error is not very helpful. I assume the error is that my instrument driver is non-pickable. I think that the error should reflect this. Is it possible to check if a driver is pickable, and if not raise a very specific and clear error to the user such that it is clear were the problem actually is? The Error I would like to get from this is something that contains the name or class of the instrument and the fact that it is unpickable. I don't know if it is possible to give more specific information on why it is non-pickable, if it were possible I would like to see that information in the error as well

Finally, related to QCoDeS/Qcodes#53 , I would like it a lot if it were possible to still run this simulation in the background for an unpickable instrument. (The problem is that Kwant systems are unpickable, such that it is not possible to write my Instrument in a pickable way)

[alexcjohnson]

Re: getting more information on what caused the pickling error - Just looking at the traceback, I don't think I'm going to be able to do much. I could wrap p.start() in a try/except just to add "this probably means something in one of your instruments is not picklable" to the error, but the error itself doesn't give me any more context to work with. Beyond that the only thing I can think of is to try and reconstruct what multiprocessing does when it collects and pickles objects for forking, and step through it to pinpoint the error. That sounds like a major headache...

Re: still running in the background - fundamentally, if you want to run this in the background, everything you will interact with in the loop needs to be somewhere outside the main process. Which means that either it's picklable so it can be sent to a separate process, or it needs to have been created in a separate process in the first place and no matter which process wants to talk to it, you do so by sending messages to it in its own process.

The latter is what I've settled on doing even for simple models, because this also means the model never gets copied, so it can have state that every caller in any process will agree about. My MockModel is probably a bit overkill for what you want to do (it was meant to mimic real instrument communication channels, so it's separate from the MockInstruments and has a stringified interface to mimic network communication channels), though it would probably work. You can see how that works now with the example system here.

Simpler, and probably better for this purpose, is to just make a single Instrument that contains the Kwant model and interacts with all of its degrees of freedom as Parameters. The key (in the branch I'm using to fix QCoDeS/Qcodes#53 ) is to do everything unpicklable (ie creating the Kwant system) in the server process, by putting it in the on_connect method, which is decorated with @ask_server (or @write_server, since this doesn't return anything, but @ask_server is safer because the main process blocks until on_connect finishes and can therefore catch any errors that happen during this setup.)

Check out how VisaInstrument does this - it stores the information it needs to create the connection in the instrument prior to starting the server (which happens in super().__init__, which we have to put at the END of __init__ rather than the beginning where it normally goes) and then all the unpicklable stuff (involving self.visa_handle) happens on the server.

Will that work for Kwant systems? You may also find the getattr and setattr methods useful - they work exactly like the getattr and setattr built-in functions if you supply a string for attr as usual, but a) they proxy these calls to the server, and b) they accept sequences for attr as well, which let you dive into nested dictionaries for flexible access of state variables.

Sorry for being so slow to respond to your issues... getting the basics of the new InstrumentServer setup working was actually pretty easy, but it's taking a long time to tweak it to be (I think) easy for users to work with, and to test it thoroughly so we don't hit any big snags like this again!

[damazter]

I think this solution might work quite well for numerical simulations. Normally you don't want the simulation to be copied or anything, because they can be very memory intensive. Hence this solution doesn't look to me as a big change to the simulation I wrote.

If I understand correctly, I need the inst-process branch in order for this to work. Would it be a good idea to start using this branch already, or shall I wait until it is merged into the master branch?

[giulioungaretti]

@damazter @alexcjohnson should we close and move forward with this ?

[alexcjohnson]

Yes, this issue is obsolete - must have gotten missed in the great purge :)