Bug with module watcher reloading when mpl_toolkit used

[philipstarkey]

Original report (archived issue) by Philip Starkey (Bitbucket: pstarkey, GitHub: philipstarkey).

---

I get a weird exception that implies that sys is None when using the following import in a lyse script (and then modifying the contents of another imported script, triggering modulewatcher to reload)

The module to import is: from mpl_toolkits.axes_grid.inset_locator import (inset_axes, InsetPosition, mark_inset).

Once you trigger module watcher to reload anything else used by the script, and re-run the analysis, you get the following traceback:

#!python

Traceback (most recent call last):
  File "y_vs_auto.py", line 24, in <module>
    from mpl_toolkits.axes_grid.inset_locator import (inset_axes, InsetPosition,
  File "C:\labscript_suite\labscript_py27\labscript_utils\double_import_denier.py", line 38, in load_module
    return imp.load_module(name, self.fp, self.pathname, self.description)
  File "C:\Anaconda3\envs\labscript_py27\lib\site-packages\mpl_toolkits\__init__.py", line 2, in <module>
    __import__('pkg_resources').declare_namespace(__name__)
  File "C:\labscript_suite\labscript_py27\labscript_utils\double_import_denier.py", line 38, in load_module
    return imp.load_module(name, self.fp, self.pathname, self.description)
  File "C:\Anaconda3\envs\labscript_py27\lib\site-packages\pkg_resources\__init__.py", line 50, in <module>
    from pkg_resources.extern import six
  File "C:\Anaconda3\envs\labscript_py27\lib\site-packages\pkg_resources\extern\__init__.py", line 44, in load_module
    mod = sys.modules[extant]
AttributeError: 'NoneType' object has no attribute 'modules'

It would be nice to get this fixed soon so that the fix can be in the version of lyse we specify to use with the data collection for @shjohnst vortex paper.

[philipstarkey]

Original comment by Chris Billington (Bitbucket: cbillington, GitHub: chrisjbillington).

---

Global variables get set to None in modules before they are garbage collected.

Looks like mpl_toolkits does something funky with its imports, which is conflicting with the magic we do.

I notice mpl_toolkits is a 'builtin' module (it has no __file__ attribute). We probably shouldn't unload these. Or maybe we aren't, but we're unloading their dependencies.

Not sure. I'll look into it. Might consider just whitelisting everything in site-packages.

[philipstarkey]

Original comment by Chris Billington (Bitbucket: cbillington, GitHub: chrisjbillington).

---

I'm not able to reproduce this, on Ubuntu at least. What versions of python, matplotlib, etc do you see this with? Windows, presumably?

[philipstarkey]

Original comment by Philip Starkey (Bitbucket: philipstarkey, GitHub: philipstarkey).

---

Hmm odd. I'm using Python 2.7, matplotlib 1.5.1 on windows.

[philipstarkey]

Original comment by Chris Billington (Bitbucket: cbillington, GitHub: chrisjbillington).

---

Well I'm using matplotlib 2.2, so would not be too surprising for it to be different. Is just upgrading a solution? The visual differences between version 1 and version 2 are pretty considerable, so maybe you don't want this as it would change the look of things with the data sharing for Shaun's paper.

Nonetheless I'll try downgrading to see if it reproduces.

[philipstarkey]

Original comment by Chris Billington (Bitbucket: cbillington, GitHub: chrisjbillington).

---

Ok, I can reproduce and I see the problem.

setuptools.pkg_resources.extern.VendorImporter, which is used by mpl_toolkits to import its submodules, adds itself to sys.meta_path (a list of objects for overriding import behaviour). The module it lives in is deleted from sys.modules, but the importer instance is still in sys.meta_path and one of its methods expects to be able to look up global variables still. That method gets called next time mpl_toolkits tries to load some submodules, but that instance sees a global scope where everything is None, and breaks.

Solution is likely to have ModuleWatcher roll back sys.meta_path the same way it rolls back sys.path. I'll look into it.

[philipstarkey]

Original comment by Chris Billington (Bitbucket: cbillington, GitHub: chrisjbillington).

---

This should be fixed by labscript_utils pull request 59.

[philipstarkey]

Original comment by Chris Billington (Bitbucket: cbillington, GitHub: chrisjbillington).

---

• changed state from "new" to "resolved"