search

Homebrew / legacy-homebrew

💀 The former home of Homebrew/homebrew (deprecated)
https://brew.sh
26.99k stars 11.36k forks source link

py2cairo segfaults #14781

Closed slyons closed 11 years ago

slyons commented 12 years ago

Cairo install log: https://gist.github.com/c14c33e6f0fd3b266729 Py2Cairo install log: https://gist.github.com/142a08e76b8280e081b7

When I try to import the cairo module, however, I get the following:

Python 2.7.3 (default, Sep  6 2012, 18:50:46) 
[GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import cairo
Fatal Python error: Interpreter not initialized (version mismatch?)
Abort trap: 6

OSX 10.7.4 with the latest XCode. I've tried both the repo version of the recipe and the modified version that's currently a pull request.

2bits commented 12 years ago

Generally you get that error when there is a mismatch between HB Python and System Python in one of your libraries, causing the header files from one python to be mixed with the libraries from the other python. That can happen if you install Homebrew python but you fail to open a new terminal tab, terminal window, or quit terminal and open it again so that your environment picks up the new python. Please gist your brew doctor and brew --config outputs. What I usually do in this situation is uninstall all my brews except python, cmake, and xz. Then I install them again. Before you do that, please paste in the output of

otool -L /usr/local/Cellar/py2cairo/1.10.0/lib/python2.7/site-packages/cairo/_cairo.so
slyons commented 12 years ago

I've tried re-opening terminal and I still get the same issue.

Brew doctor/config: https://gist.github.com/1505d78ed734899b55d5

> otool -L _cairo.so 
_cairo.so:
    /usr/local/lib/python2.7/site-packages/cairo/_cairo.so (compatibility version 0.0.0, current version 0.0.0)
    /usr/local/opt/cairo/lib/libcairo.2.dylib (compatibility version 11203.0.0, current version 11203.2.0)
    /System/Library/Frameworks/Python.framework/Versions/2.7/Python (compatibility version 2.7.0, current version 2.7.1)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 159.0.0)```
samueljohn commented 12 years ago

@slyons thanks for the report. Indeed cairo is linked to system's python. Does reinstalling cairo help?

Investigating...

samueljohn commented 12 years ago

@slyons what's the output of python-config --prefix on your system?

samueljohn commented 12 years ago

@mxcl this seems like a bug related to superenv.

Compare:

brew install py2cairo --verbose
xcrun otool -L $(brew --prefix)/Cellar/py2cairo/1.10.0/lib/python2.7/site-packages/cairo/_cairo.so

to:

brew install py2cairo --verbose --env=std
xcrun otool -L $(brew --prefix)/Cellar/py2cairo/1.10.0/lib/python2.7/site-packages/cairo/_cairo.so

You'll see that the --env=std version links correctly to the brewed python (So @slyons that's the quick work-a-round, but please leave this issue open, until we can resolve it)

samueljohn commented 12 years ago

The formula sets LINKFLAGS for waf and as I understand superenv, that should still work :-/

mxcl commented 12 years ago

Stuff like this makes me want to remove the whole of Python from Homebrew. Virtualenv works so much better.

adamv commented 12 years ago

virtualenv doesn't install new different versions of Python, just create envs of an existing one.

slyons commented 12 years ago

I would disagree. There are reasons why having a version not mandated by Apple is incredibly useful.

@samueljohn I've been reinstalling cairo every time as a precaution.

> python-config --prefix
/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7

@samueljohn I'm also confirming that your quick-fix works, although I noticed the first one produces build warnings.( https://gist.github.com/89b37cd900ac1c10b74e )

samueljohn commented 12 years ago

virtualenv fails (often) at installing modules which involve a lot of C and/or fortran code and difficult dependencies.

The problem here is the waf build system. Somehow, in superenv, the LINKFLAGS are removed again. @mxcl could it be the case that waf adds some the -L... flag to CFLAGS but superenv removes them? But I did not see it in the logs.

mxcl commented 12 years ago

This is the link link that runs for me (from ~/Library/Logs/Homebrew/cc.log which is generated when you do HOMEBREW_LOG=1 brew install py2cairo):

[/usr/local/Library/ENV/4.3/cc -sa] xcrun clang
-isystem/usr/local/include
-isystem/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.8.sdk/usr/include 
-isystem/usr/X11/include/freetype2
-isystem/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.8.sdk/usr/include/libxml2
-isystem/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.8.sdk/usr/include/apache2
-isystem/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.8.sdk/System/Library/Frameworks/Python.framework/Versions/Current/include/python2.7
-isystem/usr/X11/include -I/usr/local/opt/gettext/include
-L/usr/local/opt/gettext/lib
-L/usr/local/opt/libffi/lib
-L/usr/local/lib
-L/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.8.sdk/usr/lib -L/usr/X11/lib
--sysroot=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.8.sdk 
src/cairomodule.c.1.o src/context.c.1.o src/font.c.1.o src/path.c.1.o src/pattern.c.1.o src/matrix.c.1.o src/surface.c.1.o
-o /private/tmp/py2cairo-1.10.0-R0bF/py2cairo-1.10.0/build_directory/src/_cairo.so
-L/usr/local/Cellar/cairo/1.12.2/lib
-lcairo -lpython2.7
-L/System/Library/Frameworks/Python.framework/Versions/2.7/lib
-dynamiclib -Wl,-F. -arch x86_64 -Wl,-F. -arch x86_64 -arch x86_64
$ brew -c1
/usr/local-0.9.3 10.8.1 4.4.1-noclt clang-421 llvm-2336 /opt/X11-2.7.2
$ brew ls python
Error: No such keg: /usr/local/Cellar/python
samueljohn commented 12 years ago

@mxcl and what does xcrun otool -L $(brew --prefix)/Cellar/py2cairo/1.10.0/lib/python2.7/site-packages/cairo/_cairo.so give you then?

mxcl commented 12 years ago 
$ xcrun otool -L $(brew --prefix)/Cellar/py2cairo/1.10.0/lib/python2.7/site-packages/cairo/_cairo.so
/usr/local/Cellar/py2cairo/1.10.0/lib/python2.7/site-packages/cairo/_cairo.so:
    /usr/local/lib/python2.7/site-packages/cairo/_cairo.so (compatibility version 0.0.0, current version 0.0.0)
    /usr/local/lib/libcairo.2.dylib (compatibility version 11203.0.0, current version 11203.2.0)
    /System/Library/Frameworks/Python.framework/Versions/2.7/Python (compatibility version 2.7.0, current version 2.7.2)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 169.3.0)

But I didn't have Homebrew's Python installed. Which AFAICT is the crux of the problem.

samueljohn commented 12 years ago

I see. But still it is strange that with --env=std that waf picks up the right one, isn't it? Superenv does not do some tricks here like auto-add systems python at some point (I can remember some cmake-prefix in superenv was pointing to system's python dir)

samueljohn commented 12 years ago

I forgot the questionmark.

jacknagel commented 12 years ago

Isn't the problem here that superenv is shoving the system python headers and libraries into each compiler invocation?

mxcl commented 12 years ago

Only an include directory (from above cc.log output):

-isystem/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.8.sdk/System/Library/Frameworks/Python.framework/Versions/Current/include/python2.7
mxcl commented 12 years ago

No I'm wrong, system python is here:

-L/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.8.sdk/usr/lib ```
mxcl commented 12 years ago

OK, I have a solution that will just work for anything using superenv with or without homebrew python.

samueljohn commented 12 years ago

Hmmm it should not be an -L flag, otherwise ...

... oh realtime comments fly in ... standing by ...

slyons commented 12 years ago

I ran it both --env=std and normal with the HOMEBREW_LOG flag and I'm not seeing any difference in cc.log.

2bits commented 12 years ago

lol, epescially the part about removing python from homebrew

mxcl commented 12 years ago

@slyons stdenv doesn't generate cc.log

mxcl commented 12 years ago

I'm not sure I understand why this is broken for CLT configurations.

Man I really need to be able to ssh into one of each configuration.

samueljohn commented 12 years ago

The linker picks up the lib in the first -L<dir> it finds. If superenv really puts -L/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.8.sdk/usr/lib before the flags given by waf to the linker, that might be the explanation. If that is so, the solution might be to append the superenv-introduces -L flags instead of prepending.

mxcl commented 12 years ago

It doesn't add that path for CLT configurations though. That is only added because I'm building on NCLT.

mxcl commented 12 years ago

If that is so, the solution might be to append the superenv-introduces -L flags instead of prepending.

Yes I think this is true. We probably should do this. I just have to ensure that by doing so the order of the system -Ls doesn't get altered (by the build-tool adding them in itself).

samueljohn commented 12 years ago

The original poster (@slyons) has apparently installed the CLTs but brew doctor says he has not. So I assume superenv treats his system like NCLT (Yep, you re-wrote the Xcode/CLT detection code :-P )

From his brew --config:

Xcode: 4.4.1 => /Developer
CLT: 1.0.0.9000000000.1.1249367152

However, I run without the CLTs.

mxcl commented 12 years ago

Yep, you re-wrote the Xcode/CLT detection code :-P

Well, no. I didn't trust what was there as it is hack on hack on hack nowadays, so superenv has its own detection that reliable for the small subset of Xcode/OS X configurations that superenv supports.

CLT: 1.0.0.9000000000.1.1249367152

This is part of the reason I didn't trust it. The version numbers that come out for the CLT mean nothing to me. What I want for that version number is an Xcode version or a date like Aug-2012-release.

I know it's hard, but currently the CLT version only means one thing to me: it's (probably) installed.

mxcl commented 12 years ago

I think the very fact that config thinks he has the CLT and doctor does not is proof that this code is fragile.

jacknagel commented 12 years ago 
Xcode: 4.4.1 => /Developer
CLT: 1.0.0.9000000000.1.1249367152
GCC-4.2: build 5666
LLVM-GCC: build 2335
Clang: 2.1 build 163

Regardless, if the CLT is installed it is mis-installed because these compilers predate even Xcode 4.2.

mxcl commented 12 years ago

OK I believe the problem is that indeed superenv puts Homebrew's lib directory ahead of anything the build-tool passes in. I need to fix that.

Also I'd like to make Python stuff just-work better with Homebrew anyway. So I would like to add brewed pythons to superenv's forcible insertions. However it's such a pita, is it python2 or 3? Is it framework install or not? OMG how did it get this complicated?

mxcl commented 12 years ago

Right, I have ready a patch that:

  1. Puts build-tool passed -L flags first in the path (-Is already were first effectively because we use -isystem in superenv)
  2. However, it still removes any system Ls from the build-tools choices because we choose that order.

I apologize for this bug it was an oversight by me.

Regarding making python just work so the hack in this formula that adds -LPYTHONPATH is unecessary. We can discuss that in the solfege ticket.

mxcl commented 12 years ago

Regardless, if the CLT is installed it is mis-installed because these compilers predate even Xcode 4.2.

We really need to make brew doctor detect this stuff. It is very common. People just install Xcode-over-Xcode and we cannot expect them to know this leaves their system a mess.

jacknagel commented 12 years ago

We really need to make brew doctor detect this stuff.

It did:

Warning: Your compilers are different from the standard versions for your Xcode.
If you have Xcode 4.3 or newer, you should install the Command Line Tools for
Xcode from within Xcode's Download preferences.
Otherwise, you should reinstall Xcode.
mxcl commented 12 years ago

I suck.

2bits commented 12 years ago

The CLT installer from XCode-4.4.x often malfunctions. We have a dozen or more cases of this. It happened to me when upgrading an XCode, and there's no CLT uninstaller.

2bits commented 12 years ago

Reminds me I have to wipe my Lion. @slyons would you post your brew --config again and tell us how the CLT install went for you from within XCode Preferences please? It would help if we knew more about it.

slyons commented 12 years ago

To be honest I had completely forgotten to upgrade my CLT. I'm still not used to the app store XCode.

Here's my current config for good measure:

HOMEBREW_VERSION: 0.9.3
HEAD: 5e5f7922ff7f1a7ed6528f31c4847089e2db6600
HOMEBREW_PREFIX: /usr/local
HOMEBREW_CELLAR: /usr/local/Cellar
CPU: 8-core 64-bit sandybridge
OS X: 10.7.4-x86_64
Xcode: 4.4.1 => /Developer
CLT: 1.0.0.9000000000.1.1249367152
GCC-4.2: build 5666
LLVM-GCC: build 2335
Clang: 2.1 build 163
System Ruby: 1.8.7-357
Perl: /usr/bin/perl
Python: /usr/local/bin/python => /usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/bin/python2.7
Ruby: /usr/bin/ruby => /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/bin/ruby

I'll look into upgrading my CLT shortly.

2bits commented 12 years ago

Thanks man.

samueljohn commented 12 years ago

The CLT mess is part of why I did the effort to support Xcode-only :-)

@slyons can you brew update and try again if @mxcl's fix did it.

@mxcl thanks!

samueljohn commented 12 years ago

@mxcl oh, and CLT numbers starting with the Xcode-version number ... say "4.4..." is the one from the preference pane and the CLT numbers starting with "1.0.0..." (like the one here) is the stand alone installer. Yep, it's a mess.

samueljohn commented 12 years ago

At least on my system, I can confirm that @mxcl has fixed it.

With brewed python:

xcrun otool -L /homebrew/Cellar/py2cairo/1.10.0/lib/python2.7/site-packages/cairo/_cairo.so
/homebrew/Cellar/py2cairo/1.10.0/lib/python2.7/site-packages/cairo/_cairo.so:
    /homebrew/lib/python2.7/site-packages/cairo/_cairo.so (compatibility version 0.0.0, current version 0.0.0)
    /homebrew/lib/libcairo.2.dylib (compatibility version 11203.0.0, current version 11203.2.0)
    /homebrew/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/Python (compatibility version 2.7.0, current version 2.7.0)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 169.3.0)

import cairo works.

slyons commented 12 years ago

I can also confirm it works :) Thanks for all the help.

2bits commented 11 years ago

@mxcl It's broken again: py2cairo built against HB python is linking again system python.

$ brew list --verbose py2cairo | grep site | grep .so | xargs otool -L
/usr/local/Cellar/py2cairo/1.10.0/lib/python2.7/site-packages/cairo/_cairo.so:
    /usr/local/lib/python2.7/site-packages/cairo/_cairo.so (compatibility version 0.0.0, current version 0.0.0)
    /usr/local/lib/libcairo.2.dylib (compatibility version 11203.0.0, current version 11203.2.0)
    /System/Library/Frameworks/Python.framework/Versions/2.7/Python (compatibility version 2.7.0, current version 2.7.2)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 169.3.0)

My install log: https://gist.github.com/3789298

2bits commented 11 years ago

It looks like it finds the wrong python config lol. I guess that's because superenv takes /usr/local/bin out of the PATH? So my hack doesn't work? 

    ENV['LINKFLAGS'] = "-L#{%x(python-config --prefix).chomp}/lib"
samueljohn commented 11 years ago

Checking for program python : /usr/bin/python

@2bits, so configure finds that python first. That's as it should be, if no brewed python is installed. We can't hard code that.

Either we make the depends_on :python add the $(brew --prefix)/opt/python/bin to the PATH if it is installed or @mxcl adds $(brew --prefix)/bin to the superenv paths.

2bits commented 11 years ago

HB python is installed in the case that fails, and it's because superenv removes /usr/local/bin from your path afaik.

mxcl commented 11 years ago

Recently I made it so /usr/local/bin is removed from the PATH, this fixes a bunch of conflictory bugs. But breaks stuff like this.

One solution is env :userpaths. But a better solution is some general python solution.

Maybe superenv should add the python paths to the link and cc lines.

2bits commented 11 years ago

@mxcl env :userpaths didn't work. 

$ set | grep PATH
PATH=/usr/local/Library/ENV/4.3:/usr/local/opt/libpng/bin:/usr/local/opt/pkg-config/bin:/usr/local/opt/xz/bin:/usr/local/opt/gettext/bin:/usr/local/opt/glib/bin:/usr/local/opt/cairo/bin:/usr/X11/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin::/usr/local/share/python:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/texlive/2012/bin/x86_64-darwin:/usr/local/Library/Contributions/cmds