gdanezis / petlib

A python library that implements a number of Privacy Enhancing Technolgies
BSD 2-Clause "Simplified" License
133 stars 33 forks source link

petlib fails to compile with openssl 1.1 #13

Closed kalikaneko closed 5 years ago

kalikaneko commented 7 years ago
 Running setup.py develop for petlib
    Complete output from command /home/kali/Virtualenvs/petlib/bin/python2 -c "import setuptools, tokenize;__file__='/home/kali/leap/petlib/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" develop --no-deps:
    running develop
    running egg_info
    writing requirements to petlib.egg-info/requires.txt
    writing petlib.egg-info/PKG-INFO
    writing top-level names to petlib.egg-info/top_level.txt
    writing dependency_links to petlib.egg-info/dependency_links.txt
    reading manifest file 'petlib.egg-info/SOURCES.txt'
    writing manifest file 'petlib.egg-info/SOURCES.txt'
    running build_ext
    generating cffi module 'build/temp.linux-x86_64-2.7/petlib._petlib.c'
    creating build
    creating build/temp.linux-x86_64-2.7
    building 'petlib._petlib' extension
    creating build/temp.linux-x86_64-2.7/build
    creating build/temp.linux-x86_64-2.7/build/temp.linux-x86_64-2.7
    x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fdebug-prefix-map=/build/python2.7-HVkOs2/python2.7-2.7.13=. -fstack-protector-strong -Wformat -Werror=format-security -fPIC -I/usr/include/python2.7 -c build/temp.linux-x86_64-2.7/petlib._petlib.c -o build/temp.linux-x86_64-2.7/build/temp.linux-x86_64-2.7/petlib._petlib.o -Wno-deprecated-declarations
    build/temp.linux-x86_64-2.7/petlib._petlib.c: In function ‘hmac_ctx_size’:
    build/temp.linux-x86_64-2.7/petlib._petlib.c:462:19: error: invalid application of ‘sizeof’ to incomplete type ‘HMAC_CTX {aka struct hmac_ctx_st}’
         return sizeof(HMAC_CTX);
                       ^~~~~~~~
    build/temp.linux-x86_64-2.7/petlib._petlib.c: At top level:
    build/temp.linux-x86_64-2.7/petlib._petlib.c:466:41: error: macro "ERR_load_crypto_strings" passed 1 arguments, but takes just 0
     extern void ERR_load_crypto_strings(void);
                                             ^
    build/temp.linux-x86_64-2.7/petlib._petlib.c:468:34: error: macro "ERR_free_strings" passed 1 arguments, but takes just 0
     extern void ERR_free_strings(void);
                                      ^
    build/temp.linux-x86_64-2.7/petlib._petlib.c:535:5: warning: function declaration isn’t a prototype [-Wstrict-prototypes]
     int setup_ssl_threads() {
         ^~~~~~~~~~~~~~~~~
    build/temp.linux-x86_64-2.7/petlib._petlib.c:1037:56: error: field ‘y’ has incomplete type
     struct _cffi_align_typedef_HMAC_CTX { char x; HMAC_CTX y; };
                                                            ^
    build/temp.linux-x86_64-2.7/petlib._petlib.c: In function ‘_cffi_d_BN_init’:
    build/temp.linux-x86_64-2.7/petlib._petlib.c:1882:3: warning: implicit declaration of function ‘BN_init’ [-Wimplicit-function-declaration]
       BN_init(x0);
       ^~~~~~~
    build/temp.linux-x86_64-2.7/petlib._petlib.c: In function ‘_cffi_d_CRYPTO_lock’:
    build/temp.linux-x86_64-2.7/petlib._petlib.c:3013:3: warning: implicit declaration of function ‘CRYPTO_lock’ [-Wimplicit-function-declaration]
       CRYPTO_lock(x0, x1, x2, x3);
       ^~~~~~~~~~~
    build/temp.linux-x86_64-2.7/petlib._petlib.c: In function ‘_cffi_d_HMAC_CTX_cleanup’:
    build/temp.linux-x86_64-2.7/petlib._petlib.c:7195:3: warning: implicit declaration of function ‘HMAC_CTX_cleanup’ [-Wimplicit-function-declaration]
       HMAC_CTX_cleanup(x0);
       ^~~~~~~~~~~~~~~~
    build/temp.linux-x86_64-2.7/petlib._petlib.c: In function ‘_cffi_d_HMAC_CTX_init’:
    build/temp.linux-x86_64-2.7/petlib._petlib.c:7231:3: warning: implicit declaration of function ‘HMAC_CTX_init’ [-Wimplicit-function-declaration]
       HMAC_CTX_init(x0);
       ^~~~~~~~~~~~~
    build/temp.linux-x86_64-2.7/petlib._petlib.c: In function ‘_cffi_checkfld__ECDSA_SIG’:
    build/temp.linux-x86_64-2.7/petlib._petlib.c:7897:23: error: dereferencing pointer to incomplete type ‘ECDSA_SIG {aka struct ECDSA_SIG_st}’
       { BIGNUM * *tmp = &p->r; (void)tmp; }
                           ^~
    build/temp.linux-x86_64-2.7/petlib._petlib.c: At top level:
    build/temp.linux-x86_64-2.7/petlib._petlib.c:7900:51: error: field ‘y’ has incomplete type
     struct _cffi_align__ECDSA_SIG { char x; ECDSA_SIG y; };
                                                       ^
    build/temp.linux-x86_64-2.7/petlib._petlib.c: In function ‘_cffi_checkfld__EVP_CIPHER_CTX’:
    build/temp.linux-x86_64-2.7/petlib._petlib.c:7907:33: error: dereferencing pointer to incomplete type ‘EVP_CIPHER_CTX {aka struct evp_cipher_ctx_st}’
       { EVP_CIPHER const * *tmp = &p->cipher; (void)tmp; }
                                     ^~
    build/temp.linux-x86_64-2.7/petlib._petlib.c: At top level:
    build/temp.linux-x86_64-2.7/petlib._petlib.c:7916:61: error: field ‘y’ has incomplete type
     struct _cffi_align__EVP_CIPHER_CTX { char x; EVP_CIPHER_CTX y; };
                                                                 ^
    build/temp.linux-x86_64-2.7/petlib._petlib.c: In function ‘_cffi_checkfld__EVP_CIPHER’:
    build/temp.linux-x86_64-2.7/petlib._petlib.c:7923:12: error: dereferencing pointer to incomplete type ‘EVP_CIPHER {aka struct evp_cipher_st}’
       (void)((p->nid) | 0);  /* check that 'EVP_CIPHER.nid' is an integer */
                ^~
    build/temp.linux-x86_64-2.7/petlib._petlib.c: At top level:
    build/temp.linux-x86_64-2.7/petlib._petlib.c:7929:53: error: field ‘y’ has incomplete type
     struct _cffi_align__EVP_CIPHER { char x; EVP_CIPHER y; };
                                                         ^
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8090:3: error: invalid use of incomplete typedef ‘ECDSA_SIG {aka struct ECDSA_SIG_st}’
       { "r", offsetof(ECDSA_SIG, r),
       ^
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8093:3: error: invalid use of incomplete typedef ‘ECDSA_SIG {aka struct ECDSA_SIG_st}’
       { "s", offsetof(ECDSA_SIG, s),
       ^
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8096:3: error: invalid use of incomplete typedef ‘EVP_CIPHER_CTX {aka struct evp_cipher_ctx_st}’
       { "cipher", offsetof(EVP_CIPHER_CTX, cipher),
       ^
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8099:3: error: invalid use of incomplete typedef ‘EVP_CIPHER_CTX {aka struct evp_cipher_ctx_st}’
       { "encrypt", offsetof(EVP_CIPHER_CTX, encrypt),
       ^
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8102:3: error: invalid use of incomplete typedef ‘EVP_CIPHER_CTX {aka struct evp_cipher_ctx_st}’
       { "buf_len", offsetof(EVP_CIPHER_CTX, buf_len),
       ^
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8105:3: error: invalid use of incomplete typedef ‘EVP_CIPHER_CTX {aka struct evp_cipher_ctx_st}’
       { "num", offsetof(EVP_CIPHER_CTX, num),
       ^
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8108:3: error: invalid use of incomplete typedef ‘EVP_CIPHER_CTX {aka struct evp_cipher_ctx_st}’
       { "key_len", offsetof(EVP_CIPHER_CTX, key_len),
       ^
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8111:3: error: invalid use of incomplete typedef ‘EVP_CIPHER_CTX {aka struct evp_cipher_ctx_st}’
       { "flags", offsetof(EVP_CIPHER_CTX, flags),
       ^
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8114:3: error: invalid use of incomplete typedef ‘EVP_CIPHER_CTX {aka struct evp_cipher_ctx_st}’
       { "final_used", offsetof(EVP_CIPHER_CTX, final_used),
       ^
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8117:3: error: invalid use of incomplete typedef ‘EVP_CIPHER_CTX {aka struct evp_cipher_ctx_st}’
       { "block_mask", offsetof(EVP_CIPHER_CTX, block_mask),
       ^
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8120:3: error: invalid use of incomplete typedef ‘EVP_CIPHER {aka struct evp_cipher_st}’
       { "nid", offsetof(EVP_CIPHER, nid),
       ^
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8123:3: error: invalid use of incomplete typedef ‘EVP_CIPHER {aka struct evp_cipher_st}’
       { "block_size", offsetof(EVP_CIPHER, block_size),
       ^
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8126:3: error: invalid use of incomplete typedef ‘EVP_CIPHER {aka struct evp_cipher_st}’
       { "key_len", offsetof(EVP_CIPHER, key_len),
       ^
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8129:3: error: invalid use of incomplete typedef ‘EVP_CIPHER {aka struct evp_cipher_st}’
       { "iv_len", offsetof(EVP_CIPHER, iv_len),
       ^
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8132:3: error: invalid use of incomplete typedef ‘EVP_CIPHER {aka struct evp_cipher_st}’
       { "flags", offsetof(EVP_CIPHER, flags),
       ^
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8157:12: error: invalid application of ‘sizeof’ to incomplete type ‘HMAC_CTX {aka struct hmac_ctx_st}’
         sizeof(HMAC_CTX), offsetof(struct _cffi_align_typedef_HMAC_CTX, y), 2, 0 },
                ^~~~~~~~
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8157:5: error: initializer element is not constant
         sizeof(HMAC_CTX), offsetof(struct _cffi_align_typedef_HMAC_CTX, y), 2, 0 },
         ^~~~~~
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8157:5: note: (near initialization for ‘_cffi_struct_unions[9].size’)
    In file included from /usr/include/sched.h:28:0,
                     from /usr/include/pthread.h:23,
                     from /usr/include/openssl/crypto.h:406,
                     from /usr/include/openssl/bio.h:20,
                     from /usr/include/openssl/err.h:21,
                     from build/temp.linux-x86_64-2.7/petlib._petlib.c:435:
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8157:23: error: initializer element is not constant
         sizeof(HMAC_CTX), offsetof(struct _cffi_align_typedef_HMAC_CTX, y), 2, 0 },
                           ^
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8157:23: note: (near initialization for ‘_cffi_struct_unions[9].alignment’)
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8159:12: error: invalid application of ‘sizeof’ to incomplete type ‘ECDSA_SIG {aka struct ECDSA_SIG_st}’
         sizeof(ECDSA_SIG), offsetof(struct _cffi_align__ECDSA_SIG, y), 2, 2 },
                ^~~~~~~~~
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8159:5: error: initializer element is not constant
         sizeof(ECDSA_SIG), offsetof(struct _cffi_align__ECDSA_SIG, y), 2, 2 },
         ^~~~~~
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8159:5: note: (near initialization for ‘_cffi_struct_unions[10].size’)
    In file included from /usr/include/sched.h:28:0,
                     from /usr/include/pthread.h:23,
                     from /usr/include/openssl/crypto.h:406,
                     from /usr/include/openssl/bio.h:20,
                     from /usr/include/openssl/err.h:21,
                     from build/temp.linux-x86_64-2.7/petlib._petlib.c:435:
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8159:24: error: initializer element is not constant
         sizeof(ECDSA_SIG), offsetof(struct _cffi_align__ECDSA_SIG, y), 2, 2 },
                            ^
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8159:24: note: (near initialization for ‘_cffi_struct_unions[10].alignment’)
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8161:12: error: invalid application of ‘sizeof’ to incomplete type ‘EVP_CIPHER_CTX {aka struct evp_cipher_ctx_st}’
         sizeof(EVP_CIPHER_CTX), offsetof(struct _cffi_align__EVP_CIPHER_CTX, y), 4, 8 },
                ^~~~~~~~~~~~~~
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8161:5: error: initializer element is not constant
         sizeof(EVP_CIPHER_CTX), offsetof(struct _cffi_align__EVP_CIPHER_CTX, y), 4, 8 },
         ^~~~~~
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8161:5: note: (near initialization for ‘_cffi_struct_unions[11].size’)
    In file included from /usr/include/sched.h:28:0,
                     from /usr/include/pthread.h:23,
                     from /usr/include/openssl/crypto.h:406,
                     from /usr/include/openssl/bio.h:20,
                     from /usr/include/openssl/err.h:21,
                     from build/temp.linux-x86_64-2.7/petlib._petlib.c:435:
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8161:29: error: initializer element is not constant
         sizeof(EVP_CIPHER_CTX), offsetof(struct _cffi_align__EVP_CIPHER_CTX, y), 4, 8 },
                                 ^
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8161:29: note: (near initialization for ‘_cffi_struct_unions[11].alignment’)
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8163:12: error: invalid application of ‘sizeof’ to incomplete type ‘EVP_CIPHER {aka struct evp_cipher_st}’
         sizeof(EVP_CIPHER), offsetof(struct _cffi_align__EVP_CIPHER, y), 12, 5 },
                ^~~~~~~~~~
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8163:5: error: initializer element is not constant
         sizeof(EVP_CIPHER), offsetof(struct _cffi_align__EVP_CIPHER, y), 12, 5 },
         ^~~~~~
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8163:5: note: (near initialization for ‘_cffi_struct_unions[12].size’)
    In file included from /usr/include/sched.h:28:0,
                     from /usr/include/pthread.h:23,
                     from /usr/include/openssl/crypto.h:406,
                     from /usr/include/openssl/bio.h:20,
                     from /usr/include/openssl/err.h:21,
                     from build/temp.linux-x86_64-2.7/petlib._petlib.c:435:
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8163:25: error: initializer element is not constant
         sizeof(EVP_CIPHER), offsetof(struct _cffi_align__EVP_CIPHER, y), 12, 5 },
                             ^
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8163:25: note: (near initialization for ‘_cffi_struct_unions[12].alignment’)
    build/temp.linux-x86_64-2.7/petlib._petlib.c: In function ‘hmac_ctx_size’:
    build/temp.linux-x86_64-2.7/petlib._petlib.c:463:1: warning: control reaches end of non-void function [-Wreturn-type]
     }
     ^
    At top level:
    build/temp.linux-x86_64-2.7/petlib._petlib.c:507:13: warning: ‘_ssl_thread_locking_function’ defined but not used [-Wunused-function]
     static void _ssl_thread_locking_function(int mode, int n, const char *file,
                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
    error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

    ----------------------------------------
gdanezis commented 7 years ago

Ah yes. Currently it targets OpenSSL 1.0.x. The good people at OpenSSL have changed some of the API which breaks things a bit. There is already a port to a Dev version in a branch, and it does not take much work to port. However, what I am unsure about is the best strategy to support both, and at compile time ensure the right API is compiled.

kalikaneko commented 7 years ago

can you point me to that branch? I was having a look at porting it but cffi was beating me. what I've done in other projects is to include a switch:

 #if OPENSSL_VERSION_NUMBER >= 0x10100001L

python's cryptography has a cleaner switch, for instance:

HMAC_CTX *Cryptography_HMAC_CTX_new(void) {                                                                                              
#if CRYPTOGRAPHY_OPENSSL_110_OR_GREATER && !defined(LIBRESSL_VERSION_NUMBER)
   return HMAC_CTX_new();
#else
     /* This uses OPENSSL_zalloc in 1.1.0, which is malloc + memset */
     HMAC_CTX *ctx = (HMAC_CTX *)OPENSSL_malloc(sizeof(HMAC_CTX));
     memset(ctx, 0, sizeof(HMAC_CTX));
     return ctx;
 #endif

I'll be happy to help testing this.

gdanezis commented 7 years ago

Well, it is the only branch on this repo called OpenSSL1.1 :-). Note that some of the code depends on a patch that was not accepted for the release version, about pairings. However, the rest concerns the new API for sigs an macs.

misaakidis commented 6 years ago

In fedora-based distros there is an OpenSSL 1.0 devel compatibility package: dnf install compat-openssl10-devel --allowerasing (disclaimer: the above command will downgrade the version of openssl-devel already installed)

bogdan-kulynych commented 6 years ago

Now breaks on Ubuntu 18.04:

  Running setup.py install for petlib ... error
    Complete output from command /usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-FFTMRA/petlib/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-QLHOzd-record/install-record.txt --single-version-externally-managed --compile:
    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-2.7
    creating build/lib.linux-x86_64-2.7/petlib
    copying petlib/ecdsa.py -> build/lib.linux-x86_64-2.7/petlib
    copying petlib/bn.py -> build/lib.linux-x86_64-2.7/petlib
    copying petlib/cipher.py -> build/lib.linux-x86_64-2.7/petlib
    copying petlib/ec.py -> build/lib.linux-x86_64-2.7/petlib
    copying petlib/compile.py -> build/lib.linux-x86_64-2.7/petlib
    copying petlib/pack.py -> build/lib.linux-x86_64-2.7/petlib
    copying petlib/bindings.py -> build/lib.linux-x86_64-2.7/petlib
    copying petlib/hmac.py -> build/lib.linux-x86_64-2.7/petlib
    copying petlib/__init__.py -> build/lib.linux-x86_64-2.7/petlib
    running build_ext
    generating cffi module 'build/temp.linux-x86_64-2.7/petlib._petlib.c'
    creating build/temp.linux-x86_64-2.7
    building 'petlib._petlib' extension
    creating build/temp.linux-x86_64-2.7/build
    creating build/temp.linux-x86_64-2.7/build/temp.linux-x86_64-2.7
    x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fdebug-prefix-map=/build/python2.7-nbjU53/python2.7-2.7.15~rc1=. -fstack-protector-strong -Wformat -Werror=format-security -fPIC -I/usr/include/python2.7 -c build/temp.linux-x86_64-2.7/petlib._petlib.c -o build/temp.linux-x86_64-2.7/build/temp.linux-x86_64-2.7/petlib._petlib.o -Wno-deprecated-declarations
    build/temp.linux-x86_64-2.7/petlib._petlib.c: In function ‘hmac_ctx_size’:
    build/temp.linux-x86_64-2.7/petlib._petlib.c:521:19: error: invalid application of ‘sizeof’ to incomplete type ‘HMAC_CTX {aka struct hmac_ctx_st}’
         return sizeof(HMAC_CTX);
                       ^~~~~~~~
    build/temp.linux-x86_64-2.7/petlib._petlib.c: At top level:
    build/temp.linux-x86_64-2.7/petlib._petlib.c:525:41: error: macro "ERR_load_crypto_strings" passed 1 arguments, but takes just 0
     extern void ERR_load_crypto_strings(void);
                                             ^
    build/temp.linux-x86_64-2.7/petlib._petlib.c:527:34: error: macro "ERR_free_strings" passed 1 arguments, but takes just 0
     extern void ERR_free_strings(void);
                                      ^
    build/temp.linux-x86_64-2.7/petlib._petlib.c:594:5: warning: function declaration isn’t a prototype [-Wstrict-prototypes]
     int setup_ssl_threads() {
         ^~~~~~~~~~~~~~~~~
    build/temp.linux-x86_64-2.7/petlib._petlib.c:1096:56: error: field ‘y’ has incomplete type
     struct _cffi_align_typedef_HMAC_CTX { char x; HMAC_CTX y; };
                                                            ^
    build/temp.linux-x86_64-2.7/petlib._petlib.c: In function ‘_cffi_d_BN_init’:
    build/temp.linux-x86_64-2.7/petlib._petlib.c:1941:3: warning: implicit declaration of function ‘BN_init’; did you mean ‘BN_print’? [-Wimplicit-function-declaration]
       BN_init(x0);
       ^~~~~~~
       BN_print
    build/temp.linux-x86_64-2.7/petlib._petlib.c: In function ‘_cffi_d_CRYPTO_lock’:
    build/temp.linux-x86_64-2.7/petlib._petlib.c:3072:3: warning: implicit declaration of function ‘CRYPTO_lock’; did you mean ‘CRYPTO_free’? [-Wimplicit-function-declaration]
       CRYPTO_lock(x0, x1, x2, x3);
       ^~~~~~~~~~~
       CRYPTO_free
    build/temp.linux-x86_64-2.7/petlib._petlib.c: In function ‘_cffi_d_HMAC_CTX_cleanup’:
    build/temp.linux-x86_64-2.7/petlib._petlib.c:7254:3: warning: implicit declaration of function ‘HMAC_CTX_cleanup’; did you mean ‘HMAC_CTX_get_md’? [-Wimplicit-function-declaration]
       HMAC_CTX_cleanup(x0);
       ^~~~~~~~~~~~~~~~
       HMAC_CTX_get_md
    build/temp.linux-x86_64-2.7/petlib._petlib.c: In function ‘_cffi_d_HMAC_CTX_init’:
    build/temp.linux-x86_64-2.7/petlib._petlib.c:7290:3: warning: implicit declaration of function ‘HMAC_CTX_init’; did you mean ‘HMAC_CTX_new’? [-Wimplicit-function-declaration]
       HMAC_CTX_init(x0);
       ^~~~~~~~~~~~~
       HMAC_CTX_new
    build/temp.linux-x86_64-2.7/petlib._petlib.c: In function ‘_cffi_checkfld__ECDSA_SIG’:
    build/temp.linux-x86_64-2.7/petlib._petlib.c:7956:23: error: dereferencing pointer to incomplete type ‘ECDSA_SIG {aka struct ECDSA_SIG_st}’
       { BIGNUM * *tmp = &p->r; (void)tmp; }
                           ^~
    build/temp.linux-x86_64-2.7/petlib._petlib.c: At top level:
    build/temp.linux-x86_64-2.7/petlib._petlib.c:7959:51: error: field ‘y’ has incomplete type
     struct _cffi_align__ECDSA_SIG { char x; ECDSA_SIG y; };
                                                       ^
    build/temp.linux-x86_64-2.7/petlib._petlib.c: In function ‘_cffi_checkfld__EVP_CIPHER_CTX’:
    build/temp.linux-x86_64-2.7/petlib._petlib.c:7966:33: error: dereferencing pointer to incomplete type ‘EVP_CIPHER_CTX {aka struct evp_cipher_ctx_st}’
       { EVP_CIPHER const * *tmp = &p->cipher; (void)tmp; }
                                     ^~
    build/temp.linux-x86_64-2.7/petlib._petlib.c: At top level:
    build/temp.linux-x86_64-2.7/petlib._petlib.c:7975:61: error: field ‘y’ has incomplete type
     struct _cffi_align__EVP_CIPHER_CTX { char x; EVP_CIPHER_CTX y; };
                                                                 ^
    build/temp.linux-x86_64-2.7/petlib._petlib.c: In function ‘_cffi_checkfld__EVP_CIPHER’:
    build/temp.linux-x86_64-2.7/petlib._petlib.c:7982:12: error: dereferencing pointer to incomplete type ‘EVP_CIPHER {aka struct evp_cipher_st}’
       (void)((p->nid) | 0);  /* check that 'EVP_CIPHER.nid' is an integer */
                ^~
    build/temp.linux-x86_64-2.7/petlib._petlib.c: At top level:
    build/temp.linux-x86_64-2.7/petlib._petlib.c:7988:53: error: field ‘y’ has incomplete type
     struct _cffi_align__EVP_CIPHER { char x; EVP_CIPHER y; };
                                                         ^
    In file included from /usr/include/sched.h:29:0,
                     from /usr/include/pthread.h:23,
                     from /usr/include/openssl/crypto.h:410,
                     from /usr/include/openssl/bio.h:20,
                     from /usr/include/openssl/err.h:21,
                     from build/temp.linux-x86_64-2.7/petlib._petlib.c:494:
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8149:10: error: invalid use of incomplete typedef ‘ECDSA_SIG {aka struct ECDSA_SIG_st}’
       { "r", offsetof(ECDSA_SIG, r),
              ^
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8152:10: error: invalid use of incomplete typedef ‘ECDSA_SIG {aka struct ECDSA_SIG_st}’
       { "s", offsetof(ECDSA_SIG, s),
              ^
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8155:15: error: invalid use of incomplete typedef ‘EVP_CIPHER_CTX {aka struct evp_cipher_ctx_st}’
       { "cipher", offsetof(EVP_CIPHER_CTX, cipher),
                   ^
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8158:16: error: invalid use of incomplete typedef ‘EVP_CIPHER_CTX {aka struct evp_cipher_ctx_st}’
       { "encrypt", offsetof(EVP_CIPHER_CTX, encrypt),
                    ^
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8161:16: error: invalid use of incomplete typedef ‘EVP_CIPHER_CTX {aka struct evp_cipher_ctx_st}’
       { "buf_len", offsetof(EVP_CIPHER_CTX, buf_len),
                    ^
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8164:12: error: invalid use of incomplete typedef ‘EVP_CIPHER_CTX {aka struct evp_cipher_ctx_st}’
       { "num", offsetof(EVP_CIPHER_CTX, num),
                ^
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8167:16: error: invalid use of incomplete typedef ‘EVP_CIPHER_CTX {aka struct evp_cipher_ctx_st}’
       { "key_len", offsetof(EVP_CIPHER_CTX, key_len),
                    ^
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8170:14: error: invalid use of incomplete typedef ‘EVP_CIPHER_CTX {aka struct evp_cipher_ctx_st}’
       { "flags", offsetof(EVP_CIPHER_CTX, flags),
                  ^
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8173:19: error: invalid use of incomplete typedef ‘EVP_CIPHER_CTX {aka struct evp_cipher_ctx_st}’
       { "final_used", offsetof(EVP_CIPHER_CTX, final_used),
                       ^
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8176:19: error: invalid use of incomplete typedef ‘EVP_CIPHER_CTX {aka struct evp_cipher_ctx_st}’
       { "block_mask", offsetof(EVP_CIPHER_CTX, block_mask),
                       ^
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8179:12: error: invalid use of incomplete typedef ‘EVP_CIPHER {aka struct evp_cipher_st}’
       { "nid", offsetof(EVP_CIPHER, nid),
                ^
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8182:19: error: invalid use of incomplete typedef ‘EVP_CIPHER {aka struct evp_cipher_st}’
       { "block_size", offsetof(EVP_CIPHER, block_size),
                       ^
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8185:16: error: invalid use of incomplete typedef ‘EVP_CIPHER {aka struct evp_cipher_st}’
       { "key_len", offsetof(EVP_CIPHER, key_len),
                    ^
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8188:15: error: invalid use of incomplete typedef ‘EVP_CIPHER {aka struct evp_cipher_st}’
       { "iv_len", offsetof(EVP_CIPHER, iv_len),
                   ^
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8191:14: error: invalid use of incomplete typedef ‘EVP_CIPHER {aka struct evp_cipher_st}’
       { "flags", offsetof(EVP_CIPHER, flags),
                  ^
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8216:12: error: invalid application of ‘sizeof’ to incomplete type ‘HMAC_CTX {aka struct hmac_ctx_st}’
         sizeof(HMAC_CTX), offsetof(struct _cffi_align_typedef_HMAC_CTX, y), 2, 0 },
                ^~~~~~~~
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8216:5: warning: initialization makes integer from pointer without a cast [-Wint-conversion]
         sizeof(HMAC_CTX), offsetof(struct _cffi_align_typedef_HMAC_CTX, y), 2, 0 },
         ^~~~~~
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8216:5: note: (near initialization for ‘_cffi_struct_unions[9].size’)
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8216:5: error: initializer element is not constant
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8216:5: note: (near initialization for ‘_cffi_struct_unions[9].size’)
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8216:23: warning: initialization makes integer from pointer without a cast [-Wint-conversion]
         sizeof(HMAC_CTX), offsetof(struct _cffi_align_typedef_HMAC_CTX, y), 2, 0 },
                           ^~~~~~~~
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8216:23: note: (near initialization for ‘_cffi_struct_unions[9].alignment’)
    In file included from /usr/include/sched.h:29:0,
                     from /usr/include/pthread.h:23,
                     from /usr/include/openssl/crypto.h:410,
                     from /usr/include/openssl/bio.h:20,
                     from /usr/include/openssl/err.h:21,
                     from build/temp.linux-x86_64-2.7/petlib._petlib.c:494:
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8216:23: error: initializer element is not constant
         sizeof(HMAC_CTX), offsetof(struct _cffi_align_typedef_HMAC_CTX, y), 2, 0 },
                           ^
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8216:23: note: (near initialization for ‘_cffi_struct_unions[9].alignment’)
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8218:12: error: invalid application of ‘sizeof’ to incomplete type ‘ECDSA_SIG {aka struct ECDSA_SIG_st}’
         sizeof(ECDSA_SIG), offsetof(struct _cffi_align__ECDSA_SIG, y), 2, 2 },
                ^~~~~~~~~
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8218:5: warning: initialization makes integer from pointer without a cast [-Wint-conversion]
         sizeof(ECDSA_SIG), offsetof(struct _cffi_align__ECDSA_SIG, y), 2, 2 },
         ^~~~~~
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8218:5: note: (near initialization for ‘_cffi_struct_unions[10].size’)
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8218:5: error: initializer element is not constant
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8218:5: note: (near initialization for ‘_cffi_struct_unions[10].size’)
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8218:24: warning: initialization makes integer from pointer without a cast [-Wint-conversion]
         sizeof(ECDSA_SIG), offsetof(struct _cffi_align__ECDSA_SIG, y), 2, 2 },
                            ^~~~~~~~
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8218:24: note: (near initialization for ‘_cffi_struct_unions[10].alignment’)
    In file included from /usr/include/sched.h:29:0,
                     from /usr/include/pthread.h:23,
                     from /usr/include/openssl/crypto.h:410,
                     from /usr/include/openssl/bio.h:20,
                     from /usr/include/openssl/err.h:21,
                     from build/temp.linux-x86_64-2.7/petlib._petlib.c:494:
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8218:24: error: initializer element is not constant
         sizeof(ECDSA_SIG), offsetof(struct _cffi_align__ECDSA_SIG, y), 2, 2 },
                            ^
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8218:24: note: (near initialization for ‘_cffi_struct_unions[10].alignment’)
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8220:12: error: invalid application of ‘sizeof’ to incomplete type ‘EVP_CIPHER_CTX {aka struct evp_cipher_ctx_st}’
         sizeof(EVP_CIPHER_CTX), offsetof(struct _cffi_align__EVP_CIPHER_CTX, y), 4, 8 },
                ^~~~~~~~~~~~~~
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8220:5: warning: initialization makes integer from pointer without a cast [-Wint-conversion]
         sizeof(EVP_CIPHER_CTX), offsetof(struct _cffi_align__EVP_CIPHER_CTX, y), 4, 8 },
         ^~~~~~
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8220:5: note: (near initialization for ‘_cffi_struct_unions[11].size’)
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8220:5: error: initializer element is not constant
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8220:5: note: (near initialization for ‘_cffi_struct_unions[11].size’)
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8220:29: warning: initialization makes integer from pointer without a cast [-Wint-conversion]
         sizeof(EVP_CIPHER_CTX), offsetof(struct _cffi_align__EVP_CIPHER_CTX, y), 4, 8 },
                                 ^~~~~~~~
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8220:29: note: (near initialization for ‘_cffi_struct_unions[11].alignment’)
    In file included from /usr/include/sched.h:29:0,
                     from /usr/include/pthread.h:23,
                     from /usr/include/openssl/crypto.h:410,
                     from /usr/include/openssl/bio.h:20,
                     from /usr/include/openssl/err.h:21,
                     from build/temp.linux-x86_64-2.7/petlib._petlib.c:494:
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8220:29: error: initializer element is not constant
         sizeof(EVP_CIPHER_CTX), offsetof(struct _cffi_align__EVP_CIPHER_CTX, y), 4, 8 },
                                 ^
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8220:29: note: (near initialization for ‘_cffi_struct_unions[11].alignment’)
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8222:12: error: invalid application of ‘sizeof’ to incomplete type ‘EVP_CIPHER {aka struct evp_cipher_st}’
         sizeof(EVP_CIPHER), offsetof(struct _cffi_align__EVP_CIPHER, y), 12, 5 },
                ^~~~~~~~~~
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8222:5: warning: initialization makes integer from pointer without a cast [-Wint-conversion]
         sizeof(EVP_CIPHER), offsetof(struct _cffi_align__EVP_CIPHER, y), 12, 5 },
         ^~~~~~
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8222:5: note: (near initialization for ‘_cffi_struct_unions[12].size’)
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8222:5: error: initializer element is not constant
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8222:5: note: (near initialization for ‘_cffi_struct_unions[12].size’)
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8222:25: warning: initialization makes integer from pointer without a cast [-Wint-conversion]
         sizeof(EVP_CIPHER), offsetof(struct _cffi_align__EVP_CIPHER, y), 12, 5 },
                             ^~~~~~~~
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8222:25: note: (near initialization for ‘_cffi_struct_unions[12].alignment’)
    In file included from /usr/include/sched.h:29:0,
                     from /usr/include/pthread.h:23,
                     from /usr/include/openssl/crypto.h:410,
                     from /usr/include/openssl/bio.h:20,
                     from /usr/include/openssl/err.h:21,
                     from build/temp.linux-x86_64-2.7/petlib._petlib.c:494:
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8222:25: error: initializer element is not constant
         sizeof(EVP_CIPHER), offsetof(struct _cffi_align__EVP_CIPHER, y), 12, 5 },
                             ^
    build/temp.linux-x86_64-2.7/petlib._petlib.c:8222:25: note: (near initialization for ‘_cffi_struct_unions[12].alignment’)
    build/temp.linux-x86_64-2.7/petlib._petlib.c: In function ‘hmac_ctx_size’:
    build/temp.linux-x86_64-2.7/petlib._petlib.c:522:1: warning: control reaches end of non-void function [-Wreturn-type]
     }
     ^
    At top level:
    build/temp.linux-x86_64-2.7/petlib._petlib.c:566:13: warning: ‘_ssl_thread_locking_function’ defined but not used [-Wunused-function]
     static void _ssl_thread_locking_function(int mode, int n, const char *file,
                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
    error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
azul commented 5 years ago

I also just hit this. @gdanezis could you take a look at the pr?

lovesh commented 5 years ago

Same problem

azul commented 5 years ago

workaround for me for now was:

sudo apt install libssl1.0-dev 
lovesh commented 5 years ago

@azul This would uninstall existing (1.1) libssl-dev and don't know what that might break. I ended up installing the code from this PR and copying _cffi_src in the installed location for petlib and petlib installs. But now bplib cannot be uninstalled. This is the error i get

bplib/src/bp_g2_mult.c:236:41: error: ‘CRYPTO_LOCK_EC_PRE_COMP’ undeclared (first use in this function)
             CRYPTO_add(&pre->references, 1, CRYPTO_LOCK_EC_PRE_COMP);
                                             ^~~~~~~~~~~~~~~~~~~~~~~
    bplib/src/bp_g2_mult.c:236:41: note: each undeclared identifier is reported only once for each function it appears in
    bplib/src/bp_g2_mult.c: In function ‘g2_pre_comp_free’:
    bplib/src/bp_g2_mult.c:243:45: error: ‘CRYPTO_LOCK_EC_PRE_COMP’ undeclared (first use in this function)
             || CRYPTO_add(&pre->references, -1, CRYPTO_LOCK_EC_PRE_COMP) > 0)
                                                 ^~~~~~~~~~~~~~~~~~~~~~~
    error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
bogdan-kulynych commented 5 years ago

@lovesh Can you tell why did you have to move _cffi_src?

Installing petlib stratight from the fork should work. Can you try this command and tell if bplib still breaks?

pip install -e git+git@github.com:bogdan-kulynych/petlib.git@48a8d614743ef9396d80a76d3636749d103ddb3b#egg=petlib

The error you got is most likely not related to petlib though. It looks like the openssl version mismatch.

lovesh commented 5 years ago

@bogdan-kulynych I did not use pip but downloaded the code locally and then installed it. It was giving this error

FileNotFoundError: [Errno 2] No such file or directory: '/home/lovesh/.virtualenvs/coconut/lib/python3.5/site-packages/petlib-0.0.44-py3.5-linux-x86_64.egg/petlib/_cffi_src/openssl/openssl_v1_1.h'

So i had moved _cffi_src under petlib I removed petlib and again installed it using pip install -e git+git@github.com:bogdan-kulynych/petlib.git@48a8d614743ef9396d80a76d3636749d103ddb3b#egg=petlib but the same error

ossdev07 commented 5 years ago

@gdanezis do you have any clue how to support both version of openssl (1.0.x and 1.1.x) and resolve their dependencies at run-time ..