search

Homebrew / legacy-homebrew

💀 The former home of Homebrew/homebrew (deprecated)
https://brew.sh
26.97k stars 11.35k forks source link

uwsgi fails to build plugins logcrypto, go, v8, mongodb, postgresql, php, libffi #46507

Closed mofoyoda closed 8 years ago

mofoyoda commented 8 years ago

I'm building uwsgi with:

brew install --with-geoip --with-gloox --with-go --with-jansson --with-java --with-libffi --with-libxslt --with-libyaml --with-mongodb --with-nagios --with-php --with-postgresql --with-pypy --with-rrdtool --with-rsyslog --with-ruby --with-v8 --verbose uwsgi

My logs are here https://gist.github.com/anonymous/dcc62efc0c11c3b9470c XCode isn't latest, but it's a python exception, so I guess it doesn't matter.

apjanke commented 8 years ago

The uwsgiconfig.py build script is Python, but that's a C compilation error.

python
uwsgiconfig.py
--plugin
plugins/logcrypto
brew

plugins/logcrypto/logcrypto.c:20:2: error: unknown type name 'EVP_CIPHER_CTX'
        EVP_CIPHER_CTX *encrypt_ctx;
        ^
plugins/logcrypto/logcrypto.c:31:13: error: no member named 'ssl_initialized' in 'struct uwsgi_server'
        if (!uwsgi.ssl_initialized) {
             ~~~~~ ^
plugins/logcrypto/logcrypto.c:35:49: error: use of undeclared identifier 'EVP_CIPHER_CTX'
        uclc->encrypt_ctx = uwsgi_malloc(sizeof(EVP_CIPHER_CTX));
                                                ^
plugins/logcrypto/logcrypto.c:38:15: error: unknown type name 'EVP_CIPHER'
        const EVP_CIPHER *cipher = EVP_get_cipherbyname(uclc->algo);
              ^
plugins/logcrypto/logcrypto.c:130:52: error: use of undeclared identifier 'EVP_MAX_BLOCK_LENGTH'
        unsigned char *encrypted = uwsgi_malloc(ub->pos + EVP_MAX_BLOCK_LENGTH);
                                                          ^
5 errors generated.
using profile: buildconf/brew.ini

I can reproduce the error on my 10.11 box. Same error, same location. Build logs: https://gist.github.com/apjanke/6873002196337e2d8867

EVP_CIPHER_CTX is a constant supplied by OpenSSL. Dunno why it's not being found here.

On my 10.9.5 box, the installation gets further, erroring out on the emperor_mongodb plugin instead of logcrypto. Build logs: https://gist.github.com/13ccb1718e2cc8ba60c7

python
uwsgiconfig.py
--plugin
plugins/emperor_mongodb
brew

plugins/emperor_mongodb/emperor_mongodb.cc:3:10: fatal error: 'client/dbclient.h' file not found
#include "client/dbclient.h"
         ^
1 error generated.
using profile: buildconf/brew.ini
detected include path: ['/usr/local/include', '/Applications/Xcode-6.2.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/6.0/include', '/Applications/Xcode-6.2.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include', '/usr/include', '/System/Library/Frameworks', '/Library/Frameworks']
*** uWSGI building and linking plugin plugins/emperor_mongodb ***
[/usr/bin/clang] /usr/local/Cellar/uwsgi/2.0.11.2/libexec/uwsgi/emperor_mongodb_plugin.so
*** unable to build emperor_mongodb plugin ***
apjanke commented 8 years ago

I modified the uwsgi formula to build the plugins with --verbose, too, so we get fuller logs, including the actual clang command line used.

Updated verbose build logs from my 10.11 box: https://gist.github.com/dba15580aa9bfed381d1

Here's the failing build command.

detected include path: ['/usr/local/include', '/Applications/Xcode-7.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/7.0.0/include', '/Applications/Xcode-7.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include', '/usr/include', '/System/Library/Frameworks', '/Library/Frameworks'] * uWSGI building and linking plugin plugins/logcrypto * /usr/bin/clang -fPIC -dynamiclib -undefined dynamic_lookup -o /usr/local/Cellar/uwsgi/2.0.11.2/libexec/uwsgi/logcrypto_plugin.so -I. -O2 -I. -Wall -Werror -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I/usr/local/opt/openssl/include -Os -w -pipe -march=native -mmacosx-version-min=10.11 -arch x86_64 -fno-strict-aliasing -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -Wno-format -Wno-format-security -DUWSGI_HAS_IFADDRS -DUWSGI_ZLIB -mmacosx-version-min=10.5 -DUWSGI_LOCK_USE_OSX_SPINLOCK -DUWSGI_EVENT_USE_KQUEUE -DUWSGI_EVENT_TIMER_USE_KQUEUE -DUWSGI_EVENT_FILEMONITOR_USE_KQUEUE -I/usr/local/Cellar/pcre/8.38/include -DUWSGI_PCRE -DUWSGI_ROUTING -DUWSGI_UUID -DUWSGI_VERSION="\"2.0.11.2\"" -DUWSGI_VERSION_BASE="2" -DUWSGI_VERSION_MAJOR="0" -DUWSGI_VERSION_MINOR="11" -DUWSGI_VERSION_REVISION="2" -DUWSGI_VERSION_CUSTOM="\"\"" -DUWSGI_YAML -DUWSGI_LIBYAML -I/usr/local/Cellar/jansson/2.7/include -DUWSGI_JSON -I/usr/local/Cellar/libxml2/2.9.3/include/libxml2 -DUWSGI_XML -DUWSGI_XML_LIBXML2 -DUWSGI_PLUGIN_DIR="\"/usr/local/Cellar/uwsgi/2.0.11.2/libexec/uwsgi\"" plugins/logcrypto/logcrypto.c -L/usr/local/opt/openssl/lib -L/usr/local/opt/icu4c/lib -L/usr/local/opt/gettext/lib -L/usr/local/opt/sqlite/lib -L/usr/local/opt/readline/lib -L/usr/local/opt/libxslt/lib -L/usr/local/opt/libxml2/lib -L/usr/local/opt/libffi/lib -L/usr/local/opt/openssl/lib -L/usr/local/lib -F/usr/local/Frameworks -Wl,-headerpad_max_install_names -arch x86_64 * unable to build logcrypto plugin *

apjanke commented 8 years ago

So, here's something. That logcrypto.c plugin file pulls in a single header with #include <uwsgi.h>.

In uwsgi.h (line 357), the SSL headers are only included if UWSGI_SSL is defined.

#ifdef UWSGI_SSL
#include "openssl/conf.h"
#include "openssl/ssl.h"
#include <openssl/err.h>
#endif

That, in turn, seems to be set in uwsgiconfig.py.


        if self.get('ssl'):
            if self.get('ssl') == 'auto':
                if self.has_include('openssl/ssl.h'):
                    self.cflags.append("-DUWSGI_SSL")
                    self.libs.append('-lssl')
                    self.libs.append('-lcrypto')
                    self.gcc_list.append('core/ssl')
                    self.gcc_list.append('core/legion')
                    report['ssl'] = True
            else:
                self.cflags.append("-DUWSGI_SSL")
                self.libs.append('-lssl')
                self.libs.append('-lcrypto')
                self.gcc_list.append('core/ssl')
                self.gcc_list.append('core/legion')
                report['ssl'] = True

I don't see -DUWSGI_SSL showing up in that build command on my 10.11 box. But it does show up when I run it on 10.9.

The OpenSSL headers were removed from OS X between 10.9 and 10.11. I'll bet that its detection logic is only seeing the system OpenSSL headers even though we have a depends_on "openssl". (Maybe because it's calling /usr/bin/clang instead of just clang, so it's not picking up the Homebrew clang wrapper?)

Seems like it ought to be seeing it, since we put a -I for it in CFLAGS. But I don't think uwsgiconfig.py respects CFLAGS for the include search path. It has its own UWSGI_INCLUDES it uses.

        self.include_path = []

        if 'UWSGI_INCLUDES' in os.environ:
            self.include_path += os.environ['UWSGI_INCLUDES'].split(',')

We may need to point uwsgi at the Homebrew openssl explicitly. I think that can be done in the buildconf/brew.ini that the formula sets up.

apjanke commented 8 years ago

Yup. Putting ssl = true in the brew build profile gets the logcrypto plugin to build on 10.11, and then it goes on to fail on the emperor_mongodb plugin like it does on 10.9. emperor_pg then fails if you remove --with-mongodb.

apjanke commented 8 years ago

Hmm. I have mongodb installed using Homebrew, but it doesn't seem to include any header files.

The libpq-fe.h PostgreSQL header file needed by emperor_pg is there in /usr/local/include.

Doing ENV.prepend "UWSGI_INCLUDES", "#{HOMEBREW_PREFIX}/include" to get Homebrew's include on their search path fixes the emperor_pg build issue for me. But I don't know if that's the Right Way to do it.

After Postgres is fixed, it fails to build the ffi plugin. It looks like the ffi plugin is called libffi in this version of UWSGI. I don't know if that's a change in uwsgi, or the formula got the plugin name wrong in the first place. Changing to plugins << "libffi" if build.with? "libffi" makes it attempt to build. Then it will fail because it can't find ffi.h.

The libffi formula installs its headers under lib/libffi-3.0.13/include/ffi.h instead of just include. I don't know how formulae that use libffi are supposed to locate that.

If I remove --with-libffi then it will get further and break on the gccgo plugin.

==> python uwsgiconfig.py --verbose --plugin plugins/gccgo brew
ld: library not found for -lgo
clang: error: linker command failed with exit code 1 (use -v to see invocation)
using profile: buildconf/brew.ini
detected include path: ['/usr/local/include', '/usr/local/opt/openssl/include', '/usr/local/opt/libxml2/include', '/usr/local/opt/libxslt/include', '/usr/local/opt/readline/include', '/usr/local/opt/sqlite/include', '/usr/local/opt/gettext/include', '/usr/local/opt/icu4c/include', '/usr/local/include', '/System/Library/Frameworks/OpenGL.framework/Versions/Current/Headers', '/Applications/Xcode-7.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/7.0.0/include', '/Applications/Xcode-7.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include', '/usr/include', '/System/Library/Frameworks', '/Library/Frameworks']
*** uWSGI building and linking plugin plugins/gccgo ***
clang -fPIC -dynamiclib -undefined dynamic_lookup -o /usr/local/Cellar/uwsgi/2.0.11.2/libexec/uwsgi/gccgo_plugin.so -I. -O2 -I. -Wall -Werror -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I/usr/local/opt/openssl/include -arch x86_64 -fno-strict-aliasing -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -Wno-format -Wno-format-security -DUWSGI_HAS_IFADDRS -DUWSGI_ZLIB -mmacosx-version-min=10.5 -DUWSGI_LOCK_USE_OSX_SPINLOCK -DUWSGI_EVENT_USE_KQUEUE -DUWSGI_EVENT_TIMER_USE_KQUEUE -DUWSGI_EVENT_FILEMONITOR_USE_KQUEUE -I/usr/local/Cellar/pcre/8.38/include -DUWSGI_PCRE -DUWSGI_ROUTING -DUWSGI_UUID -DUWSGI_VERSION="\"2.0.11.2\"" -DUWSGI_VERSION_BASE="2" -DUWSGI_VERSION_MAJOR="0" -DUWSGI_VERSION_MINOR="11" -DUWSGI_VERSION_REVISION="2" -DUWSGI_VERSION_CUSTOM="\"\"" -DUWSGI_YAML -DUWSGI_LIBYAML -I/usr/local/Cellar/jansson/2.7/include -DUWSGI_JSON -DUWSGI_SSL -I/usr/local/Cellar/libxml2/2.9.3/include/libxml2 -DUWSGI_XML -DUWSGI_XML_LIBXML2 -DUWSGI_PLUGIN_DIR="\"/usr/local/Cellar/uwsgi/2.0.11.2/libexec/uwsgi\"" -g -Wno-error plugins/gccgo/gccgo_plugin.c plugins/gccgo/uwsgi.go -L/usr/local/opt/openssl/lib -arch x86_64 -lgo
*** unable to build gccgo plugin ***

I have go installed via Homebrew, but it does not provide a go.dylib or any other go library to link against.

If I remove --with-go, then it will get further and break on the php plugin.

==> python uwsgiconfig.py --verbose --plugin plugins/php brew
ld: warning: directory not found for option '-L/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.Internal.sdk/usr/lib'
ld: library not found for -lphp5
clang: error: linker command failed with exit code 1 (use -v to see invocation)
using profile: buildconf/brew.ini
detected include path: ['/usr/local/include', '/usr/local/opt/openssl/include', '/usr/local/opt/libxml2/include', '/usr/local/opt/libxslt/include', '/usr/local/opt/readline/include', '/usr/local/opt/sqlite/include', '/usr/local/opt/gettext/include', '/usr/local/opt/icu4c/include', '/usr/local/include', '/System/Library/Frameworks/OpenGL.framework/Versions/Current/Headers', '/Applications/Xcode-7.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/7.0.0/include', '/Applications/Xcode-7.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include', '/usr/include', '/System/Library/Frameworks', '/Library/Frameworks']
*** uWSGI building and linking plugin plugins/php ***
clang -fPIC -dynamiclib -undefined dynamic_lookup -o /usr/local/Cellar/uwsgi/2.0.11.2/libexec/uwsgi/php_plugin.so -I. -O2 -I. -Wall -Werror -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I/usr/local/opt/openssl/include -arch x86_64 -fno-strict-aliasing -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -Wno-format -Wno-format-security -DUWSGI_HAS_IFADDRS -DUWSGI_ZLIB -mmacosx-version-min=10.5 -DUWSGI_LOCK_USE_OSX_SPINLOCK -DUWSGI_EVENT_USE_KQUEUE -DUWSGI_EVENT_TIMER_USE_KQUEUE -DUWSGI_EVENT_FILEMONITOR_USE_KQUEUE -I/usr/local/Cellar/pcre/8.38/include -DUWSGI_PCRE -DUWSGI_ROUTING -DUWSGI_UUID -DUWSGI_VERSION="\"2.0.11.2\"" -DUWSGI_VERSION_BASE="2" -DUWSGI_VERSION_MAJOR="0" -DUWSGI_VERSION_MINOR="11" -DUWSGI_VERSION_REVISION="2" -DUWSGI_VERSION_CUSTOM="\"\"" -DUWSGI_YAML -DUWSGI_LIBYAML -I/usr/local/Cellar/jansson/2.7/include -DUWSGI_JSON -DUWSGI_SSL -I/usr/local/Cellar/libxml2/2.9.3/include/libxml2 -DUWSGI_XML -DUWSGI_XML_LIBXML2 -DUWSGI_PLUGIN_DIR="\"/usr/local/Cellar/uwsgi/2.0.11.2/libexec/uwsgi\"" -I/usr/include/php -I/usr/include/php/main -I/usr/include/php/TSRM -I/usr/include/php/Zend -I/usr/include/php/ext -I/usr/include/php/ext/date/lib -Wno-sign-compare plugins/php/php_plugin.c plugins/php/session.c -L/usr/local/opt/openssl/lib -arch x86_64 -L/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.Internal.sdk/usr/lib -lresolv -lcrypto -lssl -lcrypto -lz -lexslt -ltidy -lresolv -ledit -lncurses -lldap -llber -liconv -liconv -lpng -lz -ljpeg -lcrypto -lssl -lcrypto -lcurl -lbz2 -lz -lpcre -lcrypto -lssl -lcrypto -lm -lxml2 -lz -licucore -lm -lgssapi_krb5 -lkrb5 -lk5crypto -lcom_err -lcurl -lxml2 -lz -licucore -lm -lxml2 -lz -licucore -lm -lnetsnmp -lcrypto -lxml2 -lz -licucore -lm -lxml2 -lz -licucore -lm -lxml2 -lz -licucore -lm -lxml2 -lz -licucore -lm -lxml2 -lz -licucore -lm -lxml2 -lz -licucore -lm -lxml2 -lxslt -lphp5
*** unable to build php plugin ***

This seems to be because the system php may not be built for embedding, and there's no programmatic dependency in the uwsgi formula to express this.

$ brew info uwsgi | grep -i php
--with-php
    Compile with PHP support (PHP must be built for embedding)

I don't know how to do that. Just installing one of the php5x formulae from the homebrew/php tap didn't work. Don't see any other mentions of it in Homebrew.

Removing --with-php got the build to go further. Then it broke on the v8 plugin.

==> python uwsgiconfig.py --verbose --plugin plugins/v8 brew
In file included from plugins/v8/v8_uwsgi.cc:1:
In file included from plugins/v8/v8_uwsgi.h:2:
/usr/local/include/v8.h:353:33: error: use of undeclared identifier 'nullptr'
  V8_INLINE MaybeLocal() : val_(nullptr) {}
                                ^
/usr/local/include/v8.h:360:51: error: use of undeclared identifier 'nullptr'
  V8_INLINE bool IsEmpty() const { return val_ == nullptr; }
                                                  ^
/usr/local/include/v8.h:364:29: error: use of undeclared identifier 'nullptr'
    out->val_ = IsEmpty() ? nullptr : this->val_;
                            ^
/usr/local/include/v8.h:430:50: error: use of undeclared identifier 'nullptr'
  bool IsFirstPass() const { return callback_ != nullptr; }
                                                 ^
/usr/local/include/v8.h:794:42: error: use of undeclared identifier 'nullptr'
  V8_INLINE Global() : PersistentBase<T>(nullptr) {}
                                         ^
/usr/local/include/v8.h:819:18: error: use of undeclared identifier 'nullptr'
    other.val_ = nullptr;
                 ^
[...snip...]

That nullptr is a literal that was introduced in C++11. We might need to explicitly enable cxx11 if building the v8 plugin.

If I then remove --with-v8, the build gets all the way through and succeeds.

Summary

So, it looks like the uwsgi formula has issues with several of the optional plugins, or I'm overlooking something basic they have in common.

Problematic plugins:

apjanke commented 8 years ago

I've checked in a fix for the OpenSSL thing that was breaking the logcrypto plugin. Going to wait for feedback from some other folks before pushing fixes for the others, since I'm not sure what the right approach is for them.

Lenocam commented 8 years ago

I'm using the postgresql recipe. Here are my logs: https://gist.github.com/anonymous/e77123d6809dda2be415

MikeMcQuaid commented 8 years ago

If this is still broken: we'll accept PRs for this but we're not actively working on it at this time.