ychin commented 1 year ago

Bug report

Checklist

[X] I am confident this is a bug in CPython, not a bug in a third-party project
[X] I have searched the CPython issue tracker, and am confident this bug has not been reported before

CPython versions tested on:

3.12

Operating systems tested on:

macOS

Output from running 'python -VV' on the command line:

Python 3.12.0rc1+ (heads/3.12:ce37fbc778, Aug 25 2023, 16:39:06) [Clang 14.0.3 (clang-1403.0.22.14.1)]

A clear and concise description of the bug:

I have been trying to diagnose some issues with Vim's compilation issues (https://github.com/vim/vim/pull/12660) with its Python interface and it appears 3.12 has introduced a dependency to linkage in its header.

The way Vim dynamic Python binding works is it loads the Python DLL/dylib/so library at runtime to allows the user to provide their own Python installation. When built this way, we only compile the Python interface code, but we don't need to explicitly link against the Python library because we are just using dlopen() to load the library and necessary functions.

When compiling this way with Python 3.12, we see this error:

Undefined symbols for architecture arm64:
  "_PyBool_Type", referenced from:
      _DictionaryUpdate in if_python3.o
      _PyList_GET_SIZE in if_python3.o
      _PyTuple_GET_SIZE in if_python3.o
  "_PyLong_Type", referenced from:
      _DictionaryUpdate in if_python3.o
      _PyList_GET_SIZE in if_python3.o
      _PyTuple_GET_SIZE in if_python3.o

The issue comes from commit 7559f5fda94, which added the asserts which now reference PyLong_Type, so if you just include the header and call Py_Size(), it will now need to link against the Python lib to find the missing PyLong_Type and PyBool_Type:

static inline Py_ssize_t Py_SIZE(PyObject *ob) {
    assert(ob->ob_type != &PyLong_Type);
    assert(ob->ob_type != &PyBool_Type);
    PyVarObject *var_ob = _PyVarObject_CAST(ob);
    return var_ob->ob_size;
}

I'm filing this as a bug because it used to be the case that you could compile against Python without requiring linking against the library at build time. This seems to change the requirement.

Note that this is only the case if limited API is not used.

ychin commented 1 year ago

Hi! Just wondering if anyone had a chance to look at this yet? When I think more about it, I believe the assert being added here is actually incorrect (rather than just pulling a build dependency) if you are linking against CPython by loading the lib at runtime using dlopen(). I feel like the best solution is just to remove the assert, but I'm not sure if there are other considerations here.

iritkatriel commented 11 months ago

CC @markshannon

python / cpython