search

labscript-suite-temp-2 / lyse

lyse is an analysis framework. It coordinates the running of python analysis scripts on experiment data as it becomes availiable, updating plots in real time.
BSD 2-Clause "Simplified" License
0 stars 0 forks source link

Plots don't update after error #25

Open philipstarkey opened 7 years ago

philipstarkey commented 7 years ago

Original report (archived issue) by Jan Werkmann (Bitbucket: PhyNerd, GitHub: PhyNerd).

When there was a error in a script the plots belonging to the script won't update on subsequent runs of the script (where there is no error). To get them to update again one needs to remove the script and add it again.

philipstarkey commented 7 years ago

Original comment by Chris Billington (Bitbucket: cbillington, GitHub: chrisjbillington).

Thanks for the report. This is surprising! I'll see if I can reproduce the problem.

philipstarkey commented 7 years ago

Original comment by Jan Werkmann (Bitbucket: PhyNerd, GitHub: PhyNerd).

I'm not 100% certain this is the cause of the problem but we encounter this quite often after a error that plots won't update from then until you remove and ad the script again.

philipstarkey commented 7 years ago

Original comment by Chris Billington (Bitbucket: cbillington, GitHub: chrisjbillington).

It could be related to runmanager issue #42 (labscript-suite-temp-2/runmanager#42), I think that would cause problems - not necessarily after an error in the script.

Basically we have code (labscript_utils.modulewatcher, used by both runmanager and lyse) that unloads Python modules so that changes in those modules will be reflected when they are next imported. And it looks like it is being a bit too overzealous unloading modules. It might be unloading things required for plotting.

philipstarkey commented 7 years ago

Original comment by Chris Billington (Bitbucket: cbillington, GitHub: chrisjbillington).

I'm not able to reproduce this, unfortunately.

I can run this script:

#!python

import lyse
import numpy as np

run = lyse.Run(lyse.path)
run.save_result('x', 5)

import matplotlib.pyplot as plt

x = np.linspace(0,10,100)
plt.plot(x, np.sin(x)*np.random.rand())

assert False

plt.show()

repeatedly, unpausing analysis each time it pauses due to the assert False, and I get plots updating (visibly different because of the rand() each time) regardless of whether the AssertionError is before or after the call to plt.show().

Could you provide a minimal example that demonstrates the problem?

philipstarkey commented 7 years ago

Original comment by Jan Werkmann (Bitbucket: PhyNerd, GitHub: PhyNerd).

I will try to create one and/or wait till it happens in the lab again and then post here. But it definitely only happens after errors.

philipstarkey commented 7 years ago

Original comment by Chris Billington (Bitbucket: cbillington, GitHub: chrisjbillington).

Right, but not after all errors?

It makes sense that if something lyse-related crashes instead of your analysis code, that this will break the analysis subprocess until it is restarted. If that's the case, it's a bug and whatever is crashing should be fixed. Even if it is your code that is causing the error prior to the plots not updating, I'd be interested to see the traceback, as it might help me narrow down the cause.

philipstarkey commented 7 years ago

Original comment by Jan Werkmann (Bitbucket: PhyNerd, GitHub: PhyNerd).

That I haven't checkt till now it but your example shows that it doesn't seem to be all errors. It happened a few times now and thats why I reported it. I'll take a look at the lyse.log tomorrow when I'm in the lab and look for the traceback (we had one of those 'no more plots' errors just this afternoon)

philipstarkey commented 7 years ago

Original comment by Chris Billington (Bitbucket: cbillington, GitHub: chrisjbillington).

Sounds good.

I'll be going on holiday soon so I may not see updates here, but will look at things when I get back if they have not already been resolved.

philipstarkey commented 7 years ago

Original comment by Jan Werkmann (Bitbucket: PhyNerd, GitHub: PhyNerd).

Ok so I was finally able to reproduce the bug and narrowed it down a bit. It occurs, when a new multishot script is added to lyse and during it's first execution a new runfile gets aded (by BLACS). Plots will not appear on subsequent executions of the script even if there are no errors. If we however paused BLACS, and then add the multishot script and then unpaused BLACS it ran just fine. The error I got was:

#!

Traceback (most recent call last):
File "C:\labscript_suite\userlib\analysislib\Einzelatom\Multishot_Umpumper_Pulslaengenscan.py", line 10, in <module>
atoms0 = df['Atoms', 'Atom0']
File "C:\ProgramData\Anaconda2\lib\site-packages\pandas\core\frame.py", line 2057, in __getitem__
return self._getitem_multilevel(key)
File "C:\ProgramData\Anaconda2\lib\site-packages\pandas\core\frame.py", line 2101, in _getitem_multilevel
loc = self.columns.get_loc(key)
File "C:\ProgramData\Anaconda2\lib\site-packages\pandas\indexes\multi.py", line 1663, in get_loc
return self._engine.get_loc(key)
File "pandas\index.pyx", line 132, in pandas.index.IndexEngine.get_loc (pandas\index.c:4433)
File "pandas\index.pyx", line 154, in pandas.index.IndexEngine.get_loc (pandas\index.c:4279)
File "pandas\src\hashtable_class_helper.pxi", line 732, in pandas.hashtable.PyObjectHashTable.get_item (pandas\hashtable.c:13742)
File "pandas\src\hashtable_class_helper.pxi", line 740, in pandas.hashtable.PyObjectHashTable.get_item (pandas\hashtable.c:13696)
KeyError: ('Atoms', 'Atom0')

For context Atoms is a single shot script and Atom0 a result of it