Eliot is incompatible with PyInstaller "one-file programs"

[exarkun]

PyInstaller (https://www.pyinstaller.org/) gets sad when eliot._util.load_module is used inside a "one-file program" (https://pyinstaller.readthedocs.io/en/stable/operating-mode.html#how-the-one-file-program-works) to create a no-I/O version of the traceback module by trying to read the source file of the traceback module. In the PyInstaller execution context there is no readable source file in the filesystem.

[exarkun]

Looking at the users of load_module in Eliot, it seems like all mutations are shallow and so a shallow copy of the module might be suitable here.

This can easily be accomplished without reading the module source:

def load_module(name, original_module):
    new_module = ModuleType(name)
    new_module.__dict__.update(original_module.__dict__)
    return new_module

[exarkun]

Apparently that simple, straightforward, understandable fix appears not to work. I should have heeded the warning in the comment, I guess. I'm looking at other options.

[crwood]

A minor correction: this issue is not exclusive to PyInstaller-generated "one-file" binaries; it also occurs in "one-folder" mode.

[itamarst]

I suspect https://docs.python.org/3/library/importlib.html#importing-a-source-file-directly would work.

[itamarst]

>>> import os, importlib.util
>>> spec = importlib.util.find_spec("os")
>>> module = importlib.util.module_from_spec(spec)
>>> spec.loader.exec_module(module)
>>> module is os
False
>>> module.curdir
'.'
>>> module.environ == os.environ
True

[exarkun]

So this suggests a load_module like:

import importlib.util
def load_module(name, original_module):
    spec = importlib.util.find_spec(original_module.__name__)
    module = importlib.util.module_from_spec(spec)
    spec.loader.exec_module(module)
    module.__name__ = name
    return module

Unlike my previously suggested work-around this one actually gives you a module where, when you override globals on it, its contents respect your override. But also unlikely the previous suggestion, this is Python 3-only solution. If it works under pyinstaller and we find a Python 2 analog then together they might make a solution to this issue.

[itamarst]

I don't really care about Python 2 very much, insofar as I'm going to drop support real soon now. As such, I'd be happy to accept a workaround for Python 2 that's less ideal, e.g. not bothering copying the module at all and just risking I/O in logging codepath for people on PyInstaller on Python 2.

[itamarst]

zipimporter doesn't implement exec_module :angry: