Can't import matplotlib on linux: module '_tkinter' has no attribute '__file__'

[konstin]

Basic matplotlib usage such as import matplotlib.pyplot as plt; plt.plot(list(range(10)), list(range(10))) crashes with module '_tkinter' has no attribute '__file__'.

On system python, _tkinter is a shared library (/usr/lib/python3.8/lib-dynload/_tkinter.cpython-38-x86_64-linux-gnu.so), while in these python builds it's a builtin module which does not have __file__. matplotlib expects tkinter to be an .so it can load: https://github.com/matplotlib/matplotlib/blob/f25c2d025fa8f70d6688e7acf4b85e4eb7590299/src/_tkagg.cpp#L310-L322

To reproduce:

wget https://github.com/indygreg/python-build-standalone/releases/download/20220502/cpython-3.10.4+20220502-x86_64_v3-unknown-linux-gnu-pgo+lto-full.tar.zst
tar xf cpython-3.10.4+20220502-x86_64_v3-unknown-linux-gnu-pgo+lto-full.tar.zst
python/install/bin/python3 -m venv venv
venv/bin/pip install matplotlib
venv/bin/python -c "import matplotlib.pyplot as plt; plt.plot(list(range(10)), list(range(10)))"

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/konsti/monotrail/pbsi/venv/lib/python3.10/site-packages/matplotlib/pyplot.py", line 2769, in plot
    return gca().plot(
  File "/home/konsti/monotrail/pbsi/venv/lib/python3.10/site-packages/matplotlib/pyplot.py", line 2274, in gca
    return gcf().gca(**kwargs)
  File "/home/konsti/monotrail/pbsi/venv/lib/python3.10/site-packages/matplotlib/pyplot.py", line 867, in gcf
    return figure()
  File "/home/konsti/monotrail/pbsi/venv/lib/python3.10/site-packages/matplotlib/pyplot.py", line 808, in figure
    manager = new_figure_manager(
  File "/home/konsti/monotrail/pbsi/venv/lib/python3.10/site-packages/matplotlib/pyplot.py", line 326, in new_figure_manager
    _warn_if_gui_out_of_main_thread()
  File "/home/konsti/monotrail/pbsi/venv/lib/python3.10/site-packages/matplotlib/pyplot.py", line 316, in _warn_if_gui_out_of_main_thread
    if (_get_required_interactive_framework(_get_backend_mod())
  File "/home/konsti/monotrail/pbsi/venv/lib/python3.10/site-packages/matplotlib/pyplot.py", line 217, in _get_backend_mod
    switch_backend(dict.__getitem__(rcParams, "backend"))
  File "/home/konsti/monotrail/pbsi/venv/lib/python3.10/site-packages/matplotlib/pyplot.py", line 268, in switch_backend
    switch_backend(candidate)
  File "/home/konsti/monotrail/pbsi/venv/lib/python3.10/site-packages/matplotlib/pyplot.py", line 288, in switch_backend
    class backend_mod(matplotlib.backend_bases._Backend):
  File "/home/konsti/monotrail/pbsi/venv/lib/python3.10/site-packages/matplotlib/pyplot.py", line 289, in backend_mod
    locals().update(vars(importlib.import_module(backend_name)))
  File "/home/konsti/monotrail/pbsi/python/install/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/konsti/monotrail/pbsi/venv/lib/python3.10/site-packages/matplotlib/backends/backend_tkagg.py", line 1, in <module>
    from . import _backend_tk
  File "/home/konsti/monotrail/pbsi/venv/lib/python3.10/site-packages/matplotlib/backends/_backend_tk.py", line 24, in <module>
    from . import _tkagg
AttributeError: module '_tkinter' has no attribute '__file__'. Did you mean: '__name__'?

Running TCL_LIBRARY="$(pwd)/python/install/lib/tcl8.6" venv/bin/python -c "import matplotlib.pyplot as plt; plt.plot(list(range(10)), list(range(10)))" instead as suggested by the docs makes no difference.

EDIT: I'm mainly using the standalone builds by libloading libpython.so and launching one of the very high level c api commands, so i'd be happy about any workaround that works with libloading or importlib hackery from rust.

[indygreg]

This is similar to https://github.com/indygreg/PyOxidizer/issues/69.

My general response to this is: __file__ is documented as optional by Python: any code requiring __file__ is bugged and should instead be using a mechanism described in https://pyoxidizer.readthedocs.io/en/pyoxidizer-0.20.0/oxidized_importer_resource_files.html.

But this case is more complex than simple resource file handling. It looks like matplotlib is opening the library in order for C code to get a handle on tk function symbols. 🤯

This is highly unorthodox, as it assumes C extensions are standalone libraries (an implementation detail of the Python distribution) and that said libraries export tk symbols (again, an implementation detail).

I'll likely need to converse with a matplotlib developer on how to best support this. I would like to have a compatibility story here, as matplotlib is a popular package and I'd like it to be supported.

[konstin]

fixed in matplotlib 🎉