search

pyauth / python-pkcs11

PKCS#11/Cryptoki support for Python
MIT License
150 stars 71 forks source link

Broken by unpinned Cython 0.29.31: CYTHON_ATOMICS error: lvalue required as left operand of assignment #137

Closed space88man closed 2 years ago

space88man commented 2 years ago

TL;DR: v0.7.0 — Build from PyPI broken by Cython 0.29.31 as setup.py does not pin cython.

On PyPI v0.7.0's setup.py does not pin the version of cython, currently breaking builds as Cython is now at 0.29.31.

Previous versions of Cython (0.29,15 – 0.29.30) all work.

$gcc --version
gcc (GCC) 12.1.1 20220507 (Red Hat 12.1.1-1)

$ python3 --version
Python 3.10.5

$ python3 -m pip wheel python_pkcs11

  gcc -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python3.10 -c pkcs11/_pkcs11.c -o build/temp.linux-x86_64-3.10/pkcs11/_pkcs11.o                  
  pkcs11/_pkcs11.c: In function ‘__pyx_import_star_set’:                                    
  pkcs11/_pkcs11.c:45689:20: error: lvalue required as left operand of assignment           
  45689 |     CYTHON_ATOMICS = __Pyx_PyObject_IsTrue(o); if (unlikely((CYTHON_ATOMICS == (int)-1) && PyErr_Occurred())) __PYX_ERR(3, 26, __pyx_L2_error)                                
        |                    ^               
  error: command '/usr/bin/gcc' failed with exit code 1           
  ----------------------------------------                                                  
  ERROR: Failed building wheel for python-pkcs11

[Update] Submitted an issue to Cython as well: https://github.com/cython/cython/issues/4927

ilpadrinohack commented 2 years ago

Same here. gcc (GCC) 12.1.1 20220507 (Red Hat 12.1.1-1) Python 3.10.5

gcc -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python3.10 -c pkcs11/_pkcs11.c -o build/temp.linux-x86_64-3.10/pkcs11/_pkcs11.o

pkcs11/_pkcs11.c: En la función ‘__pyx_import_star_set’: pkcs11/_pkcs11.c:45689:20: error: se requiere un l-valor como operando izquierdo de la asignación 45689 | CYTHON_ATOMICS = __Pyx_PyObject_IsTrue(o); if (unlikely((CYTHON_ATOMICS == (int)-1) && PyErr_Occurred())) __PYX_ERR(3, 26, __pyx_L2_error)

space88man commented 2 years ago

@ilpadrinohack : you can try with a virtualenv with Cython pinned to 0.29.15; that should work.

(venv-2) [root@44b419021403 tmp]# pip install Cython==0.29.15
Collecting Cython==0.29.15
  Downloading Cython-0.29.15-py2.py3-none-any.whl (968 kB)
     |████████████████████████████████| 968 kB 15.1 MB/s 
Installing collected packages: Cython
Successfully installed Cython-0.29.15
(venv-2) [root@44b419021403 tmp]# pip wheel python-pkcs11==0.7.0
Collecting python-pkcs11==0.7.0
  Downloading python-pkcs11-0.7.0.tar.gz (106 kB)
     |████████████████████████████████| 106 kB 12.4 MB/s 
Collecting asn1crypto
  Downloading asn1crypto-1.5.1-py2.py3-none-any.whl (105 kB)
     |████████████████████████████████| 105 kB 26.8 MB/s 
  Saved ./asn1crypto-1.5.1-py2.py3-none-any.whl
Collecting cached-property
  Downloading cached_property-1.5.2-py2.py3-none-any.whl (7.6 kB)
  Saved ./cached_property-1.5.2-py2.py3-none-any.whl
Skipping asn1crypto, due to already being wheel.
Skipping cached-property, due to already being wheel.
Building wheels for collected packages: python-pkcs11
  Building wheel for python-pkcs11 (setup.py) ... done
  Created wheel for python-pkcs11: filename=python_pkcs11-0.7.0-cp39-cp39-linux_x86_64.whl size=1268825 sha256=6ad11062e79c34f044ab99b82c4db9202017cc00a159cb86fc510712de2b1247
  Stored in directory: /root/.cache/pip/wheels/8f/9b/83/03a9f544ae723155c0d89a606e57e7ad4ac8974723fd6156ff
Successfully built python-pkcs11
wietsewarendorff commented 2 years ago

Looks like I have the same problem:

x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -ffile-prefix-map=/build/python3.9-RNBry6/python3.9-3.9.2=. -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -ffile-prefix-map=/build/python3.9-RNBry6/python3.9-3.9.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/python3.9 -c pkcs11/_pkcs11.c -o build/temp.linux-x86_64-3.9/pkcs11/_pkcs11.o
  pkcs11/_pkcs11.c: In function ‘__pyx_import_star_set’:
  pkcs11/_pkcs11.c:45689:20: error: lvalue required as left operand of assignment
  45689 |     CYTHON_ATOMICS = __Pyx_PyObject_IsTrue(o); if (unlikely((CYTHON_ATOMICS == (int)-1) && PyErr_Occurred())) __PYX_ERR(3, 26, __pyx_L2_error)
        |                    ^
  error: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1
  ----------------------------------------
  ERROR: Failed building wheel for python-pkcs11
GaelC92 commented 2 years ago

The pinning of Cython to 0.29.15 solved the problem for me.

pip install Cython==0.29.15

ilpadrinohack commented 2 years ago

@space88man Does not work either:

gcc -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/root/Descargas/programacion/pruebas/venv/include -I/usr/include/python3.10 -c pkcs11/_pkcs11.c -o build/temp.linux-x86_64-cpython-310/pkcs11/_pkcs11.o

  `pkcs11/_pkcs11.c: En la función ‘__pyx_tp_dealloc_6pkcs11_7_pkcs11_AttributeList’:

  pkcs11/_pkcs11.c:43262:5: error: se requiere un l-valor como un operando de incremento
  43262 |     ++Py_REFCNT(o);`
        |     ^~
  pkcs11/_pkcs11.c:43264:5: error: se requiere un l-valor como un operando de decremento
  43264 |     --Py_REFCNT(o);
        |     ^~
  pkcs11/_pkcs11.c: En la función ‘__pyx_tp_dealloc_6pkcs11_7_pkcs11_MechanismWithParam’:
  pkcs11/_pkcs11.c:43388:5: error: se requiere un l-valor como un operando de incremento
  43388 |     ++Py_REFCNT(o);
        |     ^~
  pkcs11/_pkcs11.c:43390:5: error: se requiere un l-valor como un operando de decremento
  43390 |     --Py_REFCNT(o);
        |     ^~
  pkcs11/_pkcs11.c: En la función ‘__pyx_tp_dealloc_6pkcs11_7_pkcs11_lib’:
  pkcs11/_pkcs11.c:43506:5: error: se requiere un l-valor como un operando de incremento
  43506 |     ++Py_REFCNT(o);
        |     ^~
  pkcs11/_pkcs11.c:43508:5: error: se requiere un l-valor como un operando de decremento
  43508 |     --Py_REFCNT(o);
        |     ^~
  pkcs11/_pkcs11.c: En la función ‘__pyx_tp_dealloc_array’:
  pkcs11/_pkcs11.c:44191:5: error: se requiere un l-valor como un operando de incremento
  44191 |     ++Py_REFCNT(o);
        |     ^~
  pkcs11/_pkcs11.c:44193:5: error: se requiere un l-valor como un operando de decremento
  44193 |     --Py_REFCNT(o);
        |     ^~
  pkcs11/_pkcs11.c: En la función ‘__pyx_tp_dealloc_memoryview’:
  pkcs11/_pkcs11.c:44502:5: error: se requiere un l-valor como un operando de incremento
  44502 |     ++Py_REFCNT(o);
        |     ^~
  pkcs11/_pkcs11.c:44504:5: error: se requiere un l-valor como un operando de decremento
  44504 |     --Py_REFCNT(o);
        |     ^~
  pkcs11/_pkcs11.c: En la función ‘__pyx_tp_dealloc__memoryviewslice’:
  pkcs11/_pkcs11.c:44752:5: error: se requiere un l-valor como un operando de incremento
  44752 |     ++Py_REFCNT(o);
        |     ^~
  pkcs11/_pkcs11.c:44754:5: error: se requiere un l-valor como un operando de decremento
  44754 |     --Py_REFCNT(o);
        |     ^~
  pkcs11/_pkcs11.c: En la función ‘__Pyx_import_all_from’:
  pkcs11/_pkcs11.c:45059:13: aviso: ‘PyUnicode_AsUnicode’ es obsoleto [-Wdeprecated-declarations]
  45059 |             PyUnicode_AS_UNICODE(name)[0] == '_')
        |             ^~~~~~~~~~~~~~~~~~~~
  En el fichero incluido desde /usr/include/python3.10/unicodeobject.h:1046,
                   desde /usr/include/python3.10/Python.h:83,
                   desde pkcs11/_pkcs11.c:4:
  /usr/include/python3.10/cpython/unicodeobject.h:580:45: nota: se declara aquí
    580 | Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
        |                                             ^~~~~~~~~~~~~~~~~~~
  pkcs11/_pkcs11.c: En la función ‘__Pyx_ParseOptionalKeywords’:
  pkcs11/_pkcs11.c:50599:21: aviso: ‘_PyUnicode_get_wstr_length’ es obsoleto [-Wdeprecated-declarations]
  50599 |                     (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
        |                     ^
  /usr/include/python3.10/cpython/unicodeobject.h:446:26: nota: se declara aquí
    446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
        |                          ^~~~~~~~~~~~~~~~~~~~~~~~~~
  pkcs11/_pkcs11.c:50599:21: aviso: ‘PyUnicode_AsUnicode’ es obsoleto [-Wdeprecated-declarations]
  50599 |                     (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
        |                     ^
  /usr/include/python3.10/cpython/unicodeobject.h:580:45: nota: se declara aquí
    580 | Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
        |                                             ^~~~~~~~~~~~~~~~~~~
  pkcs11/_pkcs11.c:50599:21: aviso: ‘_PyUnicode_get_wstr_length’ es obsoleto [-Wdeprecated-declarations]
  50599 |                     (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
        |                     ^
  /usr/include/python3.10/cpython/unicodeobject.h:446:26: nota: se declara aquí
    446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
        |                          ^~~~~~~~~~~~~~~~~~~~~~~~~~
  pkcs11/_pkcs11.c:50599:21: aviso: ‘_PyUnicode_get_wstr_length’ es obsoleto [-Wdeprecated-declarations]
  50599 |                     (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
        |                     ^
  /usr/include/python3.10/cpython/unicodeobject.h:446:26: nota: se declara aquí
    446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
        |                          ^~~~~~~~~~~~~~~~~~~~~~~~~~
  pkcs11/_pkcs11.c:50599:21: aviso: ‘PyUnicode_AsUnicode’ es obsoleto [-Wdeprecated-declarations]
  50599 |                     (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
        |                     ^
  /usr/include/python3.10/cpython/unicodeobject.h:580:45: nota: se declara aquí
    580 | Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
        |                                             ^~~~~~~~~~~~~~~~~~~
  pkcs11/_pkcs11.c:50599:21: aviso: ‘_PyUnicode_get_wstr_length’ es obsoleto [-Wdeprecated-declarations]
  50599 |                     (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
        |                     ^
  /usr/include/python3.10/cpython/unicodeobject.h:446:26: nota: se declara aquí
    446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
        |                          ^~~~~~~~~~~~~~~~~~~~~~~~~~
  pkcs11/_pkcs11.c:50615:25: aviso: ‘_PyUnicode_get_wstr_length’ es obsoleto [-Wdeprecated-declarations]
  50615 |                         (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
        |                         ^
  /usr/include/python3.10/cpython/unicodeobject.h:446:26: nota: se declara aquí
    446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
        |                          ^~~~~~~~~~~~~~~~~~~~~~~~~~
  pkcs11/_pkcs11.c:50615:25: aviso: ‘PyUnicode_AsUnicode’ es obsoleto [-Wdeprecated-declarations]
  50615 |                         (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
        |                         ^
  /usr/include/python3.10/cpython/unicodeobject.h:580:45: nota: se declara aquí
    580 | Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
        |                                             ^~~~~~~~~~~~~~~~~~~
  pkcs11/_pkcs11.c:50615:25: aviso: ‘_PyUnicode_get_wstr_length’ es obsoleto [-Wdeprecated-declarations]
  50615 |                         (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
        |                         ^
  /usr/include/python3.10/cpython/unicodeobject.h:446:26: nota: se declara aquí
    446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
        |                          ^~~~~~~~~~~~~~~~~~~~~~~~~~
  pkcs11/_pkcs11.c:50615:25: aviso: ‘_PyUnicode_get_wstr_length’ es obsoleto [-Wdeprecated-declarations]
  50615 |                         (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
        |                         ^
  /usr/include/python3.10/cpython/unicodeobject.h:446:26: nota: se declara aquí
    446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
        |                          ^~~~~~~~~~~~~~~~~~~~~~~~~~
  pkcs11/_pkcs11.c:50615:25: aviso: ‘PyUnicode_AsUnicode’ es obsoleto [-Wdeprecated-declarations]
  50615 |                         (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
        |                         ^
  /usr/include/python3.10/cpython/unicodeobject.h:580:45: nota: se declara aquí
    580 | Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
        |                                             ^~~~~~~~~~~~~~~~~~~
  pkcs11/_pkcs11.c:50615:25: aviso: ‘_PyUnicode_get_wstr_length’ es obsoleto [-Wdeprecated-declarations]
  50615 |                         (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
        |                         ^
  /usr/include/python3.10/cpython/unicodeobject.h:446:26: nota: se declara aquí
    446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
        |                          ^~~~~~~~~~~~~~~~~~~~~~~~~~
  pkcs11/_pkcs11.c: En la función ‘__Pyx_decode_c_string’:
  pkcs11/_pkcs11.c:51948:9: aviso: ‘PyUnicode_FromUnicode’ es obsoleto [-Wdeprecated-declarations]
  51948 |         return PyUnicode_FromUnicode(NULL, 0);
        |         ^~~~~~
  /usr/include/python3.10/cpython/unicodeobject.h:551:42: nota: se declara aquí
    551 | Py_DEPRECATED(3.3) PyAPI_FUNC(PyObject*) PyUnicode_FromUnicode(
        |                                          ^~~~~~~~~~~~~~~~~~~~~
  pkcs11/_pkcs11.c: En la función ‘__Pyx_Coroutine_Send’:
  pkcs11/_pkcs11.c:56559:19: aviso: declaración implícita de la función ‘_PyGen_Send’; ¿quiso decir ‘_PyGen_yf’? [-Wimplicit-function-declaration]
  56559 |             ret = _PyGen_Send((PyGenObject*)yf, value == Py_None ? NULL : value);
        |                   ^~~~~~~~~~~
        |                   _PyGen_yf
  pkcs11/_pkcs11.c:56559:17: aviso: la asignación a ‘PyObject *’ {también conocido como ‘struct _object *’} desde ‘int’ crea un puntero desde un entero sin una conversión [-Wint-conversion]
  56559 |             ret = _PyGen_Send((PyGenObject*)yf, value == Py_None ? NULL : value);
        |                 ^
  pkcs11/_pkcs11.c:56564:17: aviso: la asignación a ‘PyObject *’ {también conocido como ‘struct _object *’} desde ‘int’ crea un puntero desde un entero sin una conversión [-Wint-conversion]
  56564 |             ret = _PyGen_Send((PyGenObject*)yf, value == Py_None ? NULL : value);
        |                 ^
  pkcs11/_pkcs11.c: En la función ‘__Pyx_Generator_Next’:
  pkcs11/_pkcs11.c:56648:17: aviso: la asignación a ‘PyObject *’ {también conocido como ‘struct _object *’} desde ‘int’ crea un puntero desde un entero sin una conversión [-Wint-conversion]
  56648 |             ret = _PyGen_Send((PyGenObject*)yf, NULL);
        |                 ^
  error: command '/usr/bin/gcc' failed with exit code 1
  [end of output]`

note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for python-pkcs11 Running setup.py clean for python-pkcs11 Failed to build python-pkcs11 ERROR: Failed to build one or more wheels

space88man commented 2 years ago

https://github.com/cython/cython/issues/4927#issuecomment-1197921021 Scipy also broke: https://github.com/scipy/scipy/issues/16718

ilpadrinohack commented 2 years ago

Well, with Cython 0.29.30 seems to work as suggested in your comments

wietsewarendorff commented 2 years ago

The pinning of Cython to 0.29.15 solved the problem for me.

pip install Cython==0.29.15

This works for me also! Thanks a lot!

space88man commented 2 years ago

Turns out this is due to changes from Cython 0.29.30 -> 0.29.31; I guess it will be fixed when 0.29.32 is released. It is unfortunate that I tried to build python_pkcs11 just when 0.29.31 landed.

space88man commented 2 years ago

Closing with the release of Cython 0.29.32.