lgarrison
commented
3 years ago Attaching a reproducer for completeness. Unlikely to reproduce outside of NERSC, but you never know.
```c
/* compile with (change paths for your system using python3-config):
$ gcc -lgsl -I/global/common/cori_cle7/software/python/3.8-anaconda-2020.11/include/python3.8 -I/usr/common/software/python/3.8-anaconda-2020.11/lib/python3.8/site-packages/numpy/core/include -fPIC -shared hello.c -o hello.so
on NERSC, with modules PrgEnv-intel/6.0.5, python/3.8-anaconda-2020.11, and gsl/2.5, get
lgarriso@nid00165:~/abacus-analysis/AbacusSummit/hod$ python -c 'import hello'*** Error in `python': break adjusted to free malloc space: 0x0000010000000000 ***
Fatal Python error: Aborted
basic extension code borrowed from: https://gist.github.com/physacco/2e1b52415f3a964ad2a542a99bebed8f
*/
#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION
#include
#include
#include
#include
#include
/* Now, include the numpy header*/
#include "numpy/arrayobject.h"
//Docstrings for the methods
static char module_docstring[] = "hello\n";
// Module method definitions
static PyObject* hello_world(PyObject *self, PyObject *args) {
printf("Hello, world!\n");
Py_RETURN_NONE;
}
static PyObject* hello(PyObject *self, PyObject *args) {
const char* name;
if (!PyArg_ParseTuple(args, "s", &name)) {
return NULL;
}
printf("Hello, %s!\n", name);
Py_RETURN_NONE;
}
// Method definition object for this extension, these argumens mean:
// ml_name: The name of the method
// ml_meth: Function pointer to the method implementation
// ml_flags: Flags indicating special features of this method, such as
// accepting arguments, accepting keyword arguments, being a
// class method, or being a static method of a class.
// ml_doc: Contents of this method's docstring
static PyMethodDef hello_methods[] = {
{
"hello_world", hello_world, METH_NOARGS,
"Print 'hello world' from a method defined in a C extension."
},
{
"hello", hello, METH_VARARGS,
"Print 'hello xxx' from a method defined in a C extension."
},
{NULL, NULL, 0, NULL}
};
struct module_state {
PyObject *error;
};
// Module definition
// The arguments of this structure tell Python what to call your extension,
// what it's methods are and where to look for it's method definitions
static struct PyModuleDef hello_definition = {
PyModuleDef_HEAD_INIT,
"hello",
module_docstring,
sizeof(struct module_state),
hello_methods,
//NULL,
//_countpairs_traverse,
//_countpairs_clear,
//NULL
};
// Module initialization
// Python calls this function when importing your extension. It is important
// that this function is named PyInit_[[your_module_name]] exactly, and matches
// the name keyword argument in setup.py's setup() call.
PyObject *PyInit_hello(void) {
//Py_Initialize(); // are we supposed to do this? doesn't seem to matter
PyObject *module = PyModule_Create(&hello_definition);
// comment out to avoid error
import_array();
return module;
}
```
manodeep
General information
Issue description
Running Corrfunc from Python produces errors like:
and crashes the Python process.
Resolution
At NERSC,
module unload craype-hugepages2Mwill unload the module that causes the error. Hugepages has to be activated with environment variables, and this removes the variables that do that. It's not even necessary to recompile.The issue is apparently known by Cray, but it's not clear if there's a timeline for a fix.