python3: axis: ImportError: 'interpreter' is not a built-in module

[dwrobel]

Running axis on python3 (3.7.7) on Fedora 31 generates ImportError: 'interpreter' is not a built-in module as follows:

$ linuxcnc
LINUXCNC - 2.9.0~pre0
Machine configuration directory is '/home/sw/linuxcnc/configs/my-simulated-mill'
Machine configuration file is 'my-simulated-mill.ini'
Starting LinuxCNC...
Found file(REL): ./my-simulated-mill.hal
Note: Using POSIX non-realtime
Found file(REL): ./custom.hal
note: MAXV     max: 25.000 units/sec 1500.000 units/min
note: LJOG     max: 25.000 units/sec 1500.000 units/min
note: LJOG default: 2.500 units/sec 150.000 units/min
note: jog_order='XYZ'
note: jog_invert=set()
/home/sw/projects/linuxcnc/bin/axis:3768: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
  glnav.use_pango_font(coordinate_font, 0, 128)
PYTHON: exception during 'this' export:
ImportError: 'interpreter' is not a built-in module

Interestingly it is not a fatal error as I can mill successfully 'Linuxcnc` test.

[rene-dev]

I also noticed this message, but don't know where it comes from, or what is affected by it. some tests use the interpreter module, and they seem to work fine.

[rene-dev]

according to https://github.com/python/cpython/blob/f680b517e2701e9a3859afb62628a46eccdce17c/Python/import.c#L2153 the problem could be here: https://github.com/LinuxCNC/linuxcnc/blob/934da1459a0e1ea7ac3d89bb44661fc278280a95/src/emc/pythonplugin/python_plugin.cc#L319

[jlama]

This happens when Py_IsInitialized() == 1 (e.g. 'import gcode' in a python script), thus PyImport_ExtendInittab() does nothing and the 'interpreter' module is not registered.

[satiowadahc]

Writing it down before I loose it, here's the exception: https://github.com/LinuxCNC/linuxcnc/blob/29122049fdbd4452ae07736a84aefac102184f93/src/emc/rs274ngc/rs274ngc_pre.cc#L171

[satiowadahc]

That's it! This try catch is completely redundant in python3, The module actually gets imported here: https://github.com/LinuxCNC/linuxcnc/blob/29122049fdbd4452ae07736a84aefac102184f93/src/emc/pythonplugin/python_plugin.cc#L269 So the other one should be able to be removed with a ifdef for python3.

Running tests and a merge request will be put up

[jlama]

That is not the issue. In Python3 PyImport_ExtendInittab doesn't do anything if the interpreter has already been initialized. Then the gcode module fails to import the interpreter module because it was not registered as built-in.

Instead, when Py_IsInitialized() == 1, built-in modules need to be added manually to PyImport_GetModuleDict() . Something like this:

PythonPlugin::PythonPlugin(struct _inittab *inittab)
    : status(0)
    , module_mtime(0)
    , reload_on_change(0)
    , toplevel(0)
    , abs_path(0)
    , log_level(0)
{
    if (abs_path) {
        wchar_t *program = Py_DecodeLocale(abs_path, NULL);
        Py_SetProgramName(program);
    }

    if (inittab != NULL) {
        if (!Py_IsInitialized()) {
            if (PyImport_ExtendInittab(inittab) != 0) {
                logPP(-1, "cannot extend inittab");
                status = PLUGIN_INITTAB_FAILED;
                return;
            }
        } else {
            PyObject *sys_modules = PyImport_GetModuleDict(); // borrowed

            for (int i = 0; inittab[i].name != NULL; i++) {
                struct _inittab tab = inittab[i];
                PyObject *module = tab.initfunc();
                if (module == NULL) {
                    logPP(-1, "failed to initialize built-in module '%s'", tab.name);
                    status = PLUGIN_INITTAB_FAILED;
                    return;
                }

                PyImport_AddModule(tab.name); // borrowed
                PyDict_SetItemString(sys_modules, tab.name, module);
                Py_DECREF(module);
            }
        }
    }

    Py_UnbufferedStdioFlag = 1;
    Py_Initialize();
    initialize();
}

[satiowadahc]

You would be correct, your code worked with no edits needed!

[rene-dev]

fixed by https://github.com/LinuxCNC/linuxcnc/pull/1042