Build errors in python 3.9

[hozn]

This was working fine in python 3.8, but as we're looking to update some of our base images to support python 3.9, am running into errors building cyksuid.

Specifically I am attempting to build a wheel for cyksuid. I have been able to successfully build/install cyksuid locally by explicitly passing in the --with-cython argument to setup.py build, though I'm not sure if there's any mechanism for the wheel build procedure to also use/pass this arg.

This is using the ubuntu:focal with python3.8 from dead-snakes ppa, specifically. And attempting to build latest release at time of writing (v1.0.2).

  Building wheel for cyksuid (setup.py): started
  Building wheel for cyksuid (setup.py): finished with status 'error'
  ERROR: Command errored out with exit status 1:
   command: /usr/bin/python3.9 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-wheel-55yd7qig/cyksuid_1842be1859424b2bb45858c5c8cb48e2/setup.py'"'"'; __file__='"'"'/tmp/pip-wheel-55yd7qig/cyksuid_1842be1859424b2bb45858c5c8cb48e2/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-127r1ff8
       cwd: /tmp/pip-wheel-55yd7qig/cyksuid_1842be1859424b2bb45858c5c8cb48e2/
  Complete output (144 lines):
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-3.9
  creating build/lib.linux-x86_64-3.9/cyksuid
  copying cyksuid/__init__.py -> build/lib.linux-x86_64-3.9/cyksuid
  copying cyksuid/__version__.py -> build/lib.linux-x86_64-3.9/cyksuid
  copying cyksuid/ksuid.pyx -> build/lib.linux-x86_64-3.9/cyksuid
  copying cyksuid/fast_base62.pyx -> build/lib.linux-x86_64-3.9/cyksuid
  copying cyksuid/fast_base62.pxd -> build/lib.linux-x86_64-3.9/cyksuid
  copying cyksuid/ksuid.pxd -> build/lib.linux-x86_64-3.9/cyksuid
  running build_ext
  building 'cyksuid.fast_base62' extension
  creating build/temp.linux-x86_64-3.9
  creating build/temp.linux-x86_64-3.9/cyksuid
  x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/python3.9 -c cyksuid/fast_base62.c -o build/temp.linux-x86_64-3.9/cyksuid/fast_base62.o
  x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 build/temp.linux-x86_64-3.9/cyksuid/fast_base62.o -o build/lib.linux-x86_64-3.9/cyksuid/fast_base62.cpython-39-x86_64-linux-gnu.so
  building 'cyksuid.ksuid' extension
  x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/python3.9 -c cyksuid/ksuid.c -o build/temp.linux-x86_64-3.9/cyksuid/ksuid.o
  cyksuid/ksuid.c: In function ‘__Pyx_modinit_type_init_code’:
  cyksuid/ksuid.c:4871:35: error: ‘PyTypeObject’ {aka ‘struct _typeobject’} has no member named ‘tp_print’
   4871 |   __pyx_type_7cyksuid_5ksuid_KSUID.tp_print = 0;
        |                                   ^
  cyksuid/ksuid.c: In function ‘__Pyx_ParseOptionalKeywords’:
  cyksuid/ksuid.c:5429:21: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
   5429 |                     (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
        |                     ^
  In file included from /usr/include/python3.9/unicodeobject.h:1026,
                   from /usr/include/python3.9/Python.h:97,
                   from cyksuid/ksuid.c:23:
  /usr/include/python3.9/cpython/unicodeobject.h:446:26: note: declared here
    446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
        |                          ^~~~~~~~~~~~~~~~~~~~~~~~~~
  cyksuid/ksuid.c:5429:21: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations]
   5429 |                     (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
        |                     ^
  In file included from /usr/include/python3.9/unicodeobject.h:1026,
                   from /usr/include/python3.9/Python.h:97,
                   from cyksuid/ksuid.c:23:
  /usr/include/python3.9/cpython/unicodeobject.h:580:45: note: declared here
    580 | Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
        |                                             ^~~~~~~~~~~~~~~~~~~
  cyksuid/ksuid.c:5429:21: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
   5429 |                     (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
        |                     ^
  In file included from /usr/include/python3.9/unicodeobject.h:1026,
                   from /usr/include/python3.9/Python.h:97,
                   from cyksuid/ksuid.c:23:
  /usr/include/python3.9/cpython/unicodeobject.h:446:26: note: declared here
    446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
        |                          ^~~~~~~~~~~~~~~~~~~~~~~~~~
  cyksuid/ksuid.c:5429:21: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
   5429 |                     (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
        |                     ^
  In file included from /usr/include/python3.9/unicodeobject.h:1026,
                   from /usr/include/python3.9/Python.h:97,
                   from cyksuid/ksuid.c:23:
  /usr/include/python3.9/cpython/unicodeobject.h:446:26: note: declared here
    446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
        |                          ^~~~~~~~~~~~~~~~~~~~~~~~~~
  cyksuid/ksuid.c:5429:21: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations]
   5429 |                     (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
        |                     ^
  In file included from /usr/include/python3.9/unicodeobject.h:1026,
                   from /usr/include/python3.9/Python.h:97,
                   from cyksuid/ksuid.c:23:
  /usr/include/python3.9/cpython/unicodeobject.h:580:45: note: declared here
    580 | Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
        |                                             ^~~~~~~~~~~~~~~~~~~
  cyksuid/ksuid.c:5429:21: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
   5429 |                     (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
        |                     ^
  In file included from /usr/include/python3.9/unicodeobject.h:1026,
                   from /usr/include/python3.9/Python.h:97,
                   from cyksuid/ksuid.c:23:
  /usr/include/python3.9/cpython/unicodeobject.h:446:26: note: declared here
    446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
        |                          ^~~~~~~~~~~~~~~~~~~~~~~~~~
  cyksuid/ksuid.c:5445:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
   5445 |                         (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
        |                         ^
  In file included from /usr/include/python3.9/unicodeobject.h:1026,
                   from /usr/include/python3.9/Python.h:97,
                   from cyksuid/ksuid.c:23:
  /usr/include/python3.9/cpython/unicodeobject.h:446:26: note: declared here
    446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
        |                          ^~~~~~~~~~~~~~~~~~~~~~~~~~
  cyksuid/ksuid.c:5445:25: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations]
   5445 |                         (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
        |                         ^
  In file included from /usr/include/python3.9/unicodeobject.h:1026,
                   from /usr/include/python3.9/Python.h:97,
                   from cyksuid/ksuid.c:23:
  /usr/include/python3.9/cpython/unicodeobject.h:580:45: note: declared here
    580 | Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
        |                                             ^~~~~~~~~~~~~~~~~~~
  cyksuid/ksuid.c:5445:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
   5445 |                         (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
        |                         ^
  In file included from /usr/include/python3.9/unicodeobject.h:1026,
                   from /usr/include/python3.9/Python.h:97,
                   from cyksuid/ksuid.c:23:
  /usr/include/python3.9/cpython/unicodeobject.h:446:26: note: declared here
    446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
        |                          ^~~~~~~~~~~~~~~~~~~~~~~~~~
  cyksuid/ksuid.c:5445:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
   5445 |                         (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
        |                         ^
  In file included from /usr/include/python3.9/unicodeobject.h:1026,
                   from /usr/include/python3.9/Python.h:97,
                   from cyksuid/ksuid.c:23:
  /usr/include/python3.9/cpython/unicodeobject.h:446:26: note: declared here
    446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
        |                          ^~~~~~~~~~~~~~~~~~~~~~~~~~
  cyksuid/ksuid.c:5445:25: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations]
   5445 |                         (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
        |                         ^
  In file included from /usr/include/python3.9/unicodeobject.h:1026,
                   from /usr/include/python3.9/Python.h:97,
                   from cyksuid/ksuid.c:23:
  /usr/include/python3.9/cpython/unicodeobject.h:580:45: note: declared here
    580 | Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
        |                                             ^~~~~~~~~~~~~~~~~~~
  cyksuid/ksuid.c:5445:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
   5445 |                         (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
        |                         ^
  In file included from /usr/include/python3.9/unicodeobject.h:1026,
                   from /usr/include/python3.9/Python.h:97,
                   from cyksuid/ksuid.c:23:
  /usr/include/python3.9/cpython/unicodeobject.h:446:26: note: declared here
    446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
        |                          ^~~~~~~~~~~~~~~~~~~~~~~~~~
  cyksuid/ksuid.c: In function ‘__Pyx_decode_c_bytes’:
  cyksuid/ksuid.c:6139:9: warning: ‘PyUnicode_FromUnicode’ is deprecated [-Wdeprecated-declarations]
   6139 |         return PyUnicode_FromUnicode(NULL, 0);
        |         ^~~~~~
  In file included from /usr/include/python3.9/unicodeobject.h:1026,
                   from /usr/include/python3.9/Python.h:97,
                   from cyksuid/ksuid.c:23:
  /usr/include/python3.9/cpython/unicodeobject.h:551:42: note: declared here
    551 | Py_DEPRECATED(3.3) PyAPI_FUNC(PyObject*) PyUnicode_FromUnicode(
        |                                          ^~~~~~~~~~~~~~~~~~~~~
  error: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1
  ----------------------------------------
  ERROR: Failed building wheel for cyksuid

[timonwong]

Thanks for reporting, I'll release a new binary wheel with python3.9.

For source install, maybe it's better to have cython installed and build automatically under these circumstances

[hozn]

Thank you! Yes, in interim we will just build from source with cython as that does indeed work.

[timonwong]

3.9 wheels are uploaded as of v1.1.0 now :)

[hozn]

Thanks so much! Will go update our base images to use those.