search

msgpack / msgpack-python

MessagePack serializer implementation for Python msgpack.org[Python]
https://msgpack.org/
Other
1.91k stars 229 forks source link

Python 3.13: compile error on Ubuntu 20.04 #593

Closed tristanlatr closed 5 months ago

tristanlatr commented 5 months ago

Impossible to build msgpack on python 3.13 because of what looks like an deprecation warning

  × Building wheel for msgpack (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [45 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-cpython-313
      creating build/lib.linux-x86_64-cpython-313/msgpack
      copying msgpack/fallback.py -> build/lib.linux-x86_64-cpython-313/msgpack
      copying msgpack/__init__.py -> build/lib.linux-x86_64-cpython-313/msgpack
      copying msgpack/exceptions.py -> build/lib.linux-x86_64-cpython-313/msgpack
      copying msgpack/ext.py -> build/lib.linux-x86_64-cpython-313/msgpack
      running build_ext
      building 'msgpack._cmsgpack' extension
      creating build/temp.linux-x86_64-cpython-313
      creating build/temp.linux-x86_64-cpython-313/msgpack
      gcc -pthread -fno-strict-overflow -Wsign-compare -DNDEBUG -g -O3 -Wall -fPIC -I. -I/home/runner/work/pydoctor/pydoctor/.tox/test/include -I/opt/hostedtoolcache/Python/3.13.0-alpha.6/x64/include/python3.13 -c msgpack/_cmsgpack.cpp -o build/temp.linux-x86_64-cpython-313/msgpack/_cmsgpack.o
      msgpack/_cmsgpack.cpp:1366:72: warning: ‘Py_UNICODE’ is deprecated [-Wdeprecated-declarations]
       1366 | static CYTHON_INLINE size_t __Pyx_Py_UNICODE_strlen(const Py_UNICODE *u)
            |                                                                        ^
      In file included from /opt/hostedtoolcache/Python/3.13.0-alpha.6/x64/include/python3.13/unicodeobject.h:1014,
                       from /opt/hostedtoolcache/Python/3.13.0-alpha.6/x64/include/python3.13/Python.h:65,
                       from msgpack/_cmsgpack.cpp:16:
      /opt/hostedtoolcache/Python/3.13.0-alpha.6/x64/include/python3.13/cpython/unicodeobject.h:10:37: note: declared here
         10 | Py_DEPRECATED(3.13) typedef wchar_t Py_UNICODE;
            |                                     ^~~~~~~~~~
      msgpack/_cmsgpack.cpp: In function ‘size_t __Pyx_Py_UNICODE_strlen(const Py_UNICODE*)’:
      msgpack/_cmsgpack.cpp:1368:23: warning: ‘Py_UNICODE’ is deprecated [-Wdeprecated-declarations]
       1368 |     const Py_UNICODE *u_end = u;
            |                       ^~~~~
      In file included from /opt/hostedtoolcache/Python/3.13.0-alpha.6/x64/include/python3.13/unicodeobject.h:1014,
                       from /opt/hostedtoolcache/Python/3.13.0-alpha.6/x64/include/python3.13/Python.h:65,
                       from msgpack/_cmsgpack.cpp:16:
      /opt/hostedtoolcache/Python/3.13.0-alpha.6/x64/include/python3.13/cpython/unicodeobject.h:10:37: note: declared here
         10 | Py_DEPRECATED(3.13) typedef wchar_t Py_UNICODE;
            |                                     ^~~~~~~~~~
      In file included from /opt/hostedtoolcache/Python/3.13.0-alpha.6/x64/include/python3.13/internal/pycore_code.h:461,
                       from /opt/hostedtoolcache/Python/3.13.0-alpha.6/x64/include/python3.13/internal/pycore_frame.h:13,
                       from msgpack/_cmsgpack.cpp:22559:
      /opt/hostedtoolcache/Python/3.13.0-alpha.6/x64/include/python3.13/internal/pycore_backoff.h: In function ‘_Py_BackoffCounter make_backoff_counter(uint16_t, uint16_t)’:
      /opt/hostedtoolcache/Python/3.13.0-alpha.6/x64/include/python3.13/internal/pycore_backoff.h:47:44: warning: narrowing conversion of ‘backoff’ from ‘uint16_t’ {aka ‘short unsigned int’} to ‘unsigned char:4’ [-Wnarrowing]
         47 |     return (_Py_BackoffCounter){.backoff = backoff, .value = value};
            |                                            ^~~~~~~
      /opt/hostedtoolcache/Python/3.13.0-alpha.6/x64/include/python3.13/internal/pycore_backoff.h:47:67: error: too many initializers for ‘_Py_BackoffCounter’
         47 |     return (_Py_BackoffCounter){.backoff = backoff, .value = value};
            |                                                                   ^
      error: command '/usr/bin/gcc' failed with exit code 1
      [end of output]
tristanlatr commented 5 months ago

It started failing at 3.13.0-alpha.6, everything works on 3.13.0-alpha.5.

ThomasWaldmann commented 5 months ago

I suspect that the unicode deprecation warning does not fail the build, it is just a warning.

But that is an error:

      In file included from /opt/hostedtoolcache/Python/3.13.0-alpha.6/x64/include/python3.13/internal/pycore_code.h:461,
                       from /opt/hostedtoolcache/Python/3.13.0-alpha.6/x64/include/python3.13/internal/pycore_frame.h:13,
                       from msgpack/_cmsgpack.cpp:22559:
      /opt/hostedtoolcache/Python/3.13.0-alpha.6/x64/include/python3.13/internal/pycore_backoff.h: In function ‘_Py_BackoffCounter make_backoff_counter(uint16_t, uint16_t)’:
      /opt/hostedtoolcache/Python/3.13.0-alpha.6/x64/include/python3.13/internal/pycore_backoff.h:47:44: warning: narrowing conversion of ‘backoff’ from ‘uint16_t’ {aka ‘short unsigned int’} to ‘unsigned char:4’ [-Wnarrowing]
         47 |     return (_Py_BackoffCounter){.backoff = backoff, .value = value};
            |                                            ^~~~~~~
      /opt/hostedtoolcache/Python/3.13.0-alpha.6/x64/include/python3.13/internal/pycore_backoff.h:47:67: error: too many initializers for ‘_Py_BackoffCounter’
         47 |     return (_Py_BackoffCounter){.backoff = backoff, .value = value};
            |                                                                   ^
      error: command '/usr/bin/gcc' failed with exit code 1

Considering this is built from Cython source (.pyx), maybe the Cython version used to cythonize that does not yet support that Python version.

methane commented 5 months ago

This is not relating to Cython. Maybe, default g++ in ubuntu-20.04 Action Runner doesn't support compiling this header file.

methane commented 5 months ago

Uh, this is relating to Cython, because Cython depending on Python internal too much. Cython imports pycore_frame for traceback support.

/* AddTraceback */
#include "compile.h"
#include "frameobject.h"
#include "traceback.h"
#if PY_VERSION_HEX >= 0x030b00a6 && !CYTHON_COMPILING_IN_LIMITED_API
  #ifndef Py_BUILD_CORE
    #define Py_BUILD_CORE 1
  #endif
  #include "internal/pycore_frame.h"
#endif
#if CYTHON_COMPILING_IN_LIMITED_API
static PyObject *__Pyx_PyCode_Replace_For_AddTraceback(PyObject *code, PyObject *scratch_dict,
                                                       PyObject *firstlineno, PyObject *name) {
    PyObject *replace = NULL;
...

It cause compile error on old g++.

inada-n@ubuntu20:~$ cat foo.cpp
#define Py_BUILD_CORE 1
#include <Python.h>

#include "frameobject.h"
#include "internal/pycore_frame.h"

int main() {}

inada-n@ubuntu20:~$ g++-10 -std=c++17 -I include/python3.13 foo.cpp
In file included from include/python3.13/internal/pycore_code.h:461,
                 from include/python3.13/internal/pycore_frame.h:13,
                 from foo.cpp:5:
include/python3.13/internal/pycore_backoff.h: In function ‘_Py_BackoffCounter make_backoff_counter(uint16_t, uint16_t)’:
include/python3.13/internal/pycore_backoff.h:47:44: warning: narrowing conversion of ‘backoff’ from ‘uint16_t’ {aka ‘short unsigned int’} to ‘unsigned char:4’ [-Wnarrowing]
   47 |     return (_Py_BackoffCounter){.backoff = backoff, .value = value};
      |                                            ^~~~~~~
include/python3.13/internal/pycore_backoff.h:47:67: error: too many initializers for ‘_Py_BackoffCounter’
   47 |     return (_Py_BackoffCounter){.backoff = backoff, .value = value};
      |                                                                   ^

On Ubuntu 22.04, this error doesn't happen. Ubuntu 22.04 uses gcc 13.

methane commented 5 months ago

There is nothing we can do here. You can: