[BUG] A dynamic link library (DLL) initialization routine failed when statically invoking globals() with Python 3.8

[fteicht]

Issue description

Saving pybind11::globals() in a class static variable throws a Python interpreter' s exception under Windows 10 with Python 3.8 but it works with Python 3.7. I compile my code with MSVC 19.27.29111.0 (Visual Studio 16 2019 build tools called from cmake).

The exception is: ImportError: DLL load failed while importing __pybind11_debug: A dynamic link library (DLL) initialization routine failed.

Reproducible example code

Here is a minimal code that crashes on Windows 10 with Python 3.8 (using the command line C:\Users\vagrant\AppData\Local\Programs\Python\Python38\python.exe -c "import __pybind11_debug"):

#include <pybind11/pybind11.h>

namespace py = pybind11;

struct A {
    inline static py::object globals = py::globals();
};

PYBIND11_MODULE(__pybind11_debug, m) {
     py::class_<A> py_a(m, "A");
}

The same code perfectly works with Python 3.7 on Windows 10 (and also with Python 3.7 and 3.8 on Linux and MacOS).

Note: storing pybind11::globals() in a class static variable might seem silly and useless. I actually need to store py::globals()["__builtins__"]["NotImplemented"] that my real code calls millions of time (thus I want to save millions of dictionary key searches).

[henryiii]

Slightly related note: we don't run CI on Python 3.8+ and Windows, due to a change in importing that we need to solve (in the CI setup). I'm hoping we will get CI turned on for 3.8/3.9rc2 soon, I just haven't had time to figure it out and no one else has volunteered to fix it yet. ;)

[YannickJadoul]

@fteicht I don't immediately know how to debug this. Does a debug session give more information? Microsoft seems to have a few things that you can try: https://docs.microsoft.com/en-us/archive/blogs/tess/unable-to-load-dll-dllname-dll-a-dynamic-link-library-dll-initialization-routine-failed-0x8007045a

Apart from that, how about moving the problem to something like

inline py::object get_globals() {
    static py::object globals = py::globals();
    return globals;
}

or

inline py::object get_globals() {
    static py::object globals;
    if (!globals)
        globals = py::globals();
    return globals;
}

This is of course just moving the problem to a later time, but maybe it helps to postpone the creation of this global variable and calling into Python?

[fteicht]

Thanks a lot @YannickJadoul : your first proposed solution works like a charm. So meantime this solution answers my need but I think we should not close this issue because that's still a bug in my humble opinion. It is apparently related to the order of static variables initialization by the C++ standard library with regard to the Python interpreter initialization routines. I think we should understand why this happens only with Python 3.8 on Windows platforms so we can either recommend to not statically initialize some Python-related variables from C++ or to appropriately correct this bug. Thanks again for your great support!

[YannickJadoul]

Thanks a lot @YannickJadoul : your first proposed solution works like a charm.

Ah, good :-)

that's still a bug in my humble opinion.

I don't think there's a lot we can do about it, as far as I know. The "static initialization order fiasco" is a long-standing C++ problem (https://isocpp.org/wiki/faq/ctors#static-init-order), but as far as I can see, this does not depend on pybind11's global/static variables :-(

If anyone has a solution (or can point to somewhere that pybind11 can improve), I'd be very happy to hear about it and try improving pybind11, but otherwise, I'd suggest closing this issue? :-/