FreeBSD 7.1 unladen-2009Q3 build fails

[GoogleCodeExporter]

After using nagy.attila's suggested fix for the initial FreeBSD problems I
can compile most of unladen-2009Q3. I used this configure

and get this error

g++ -pthread -c -IUtil/llvm/include -I./Util/llvm/include
-D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -Woverloaded-virtual
-fno-strict-aliasing -fwrapv -Wall -DNDEBUG -g -O3  -I. -IInclude -I.
-I./Include   -DPy_BUILD_CORE -Wno-write-strings -o Util/PyAliasAnalysis.o
Util/PyAliasAnalysis.cc
In file included from /usr/include/c++/4.2/ios:47,
                 from /usr/include/c++/4.2/ostream:45,
                 from /usr/include/c++/4.2/iterator:70,
                 from Util/llvm/include/llvm/ADT/DenseMap.h:20,
                 from ./Util/ConstantMirror.h:10,
                 from ./Python/global_llvm_data.h:15,
                 from ./Util/PyTypeBuilder.h:14,
                 from Util/PyAliasAnalysis.cc:3:
/usr/include/c++/4.2/bits/localefwd.h:58:34: error: macro "isspace" passed
2 arguments, but takes just 1
/usr/include/c++/4.2/bits/localefwd.h:70:34: error: macro "isupper" passed
2 arguments, but takes just 1
/usr/include/c++/4.2/bits/localefwd.h:74:34: error: macro "islower" passed
2 arguments, but takes just 1
/usr/include/c++/4.2/bits/localefwd.h:78:34: error: macro "isalpha" passed
2 arguments, but takes just 1
/usr/include/c++/4.2/bits/localefwd.h:94:34: error: macro "isalnum" passed
2 arguments, but takes just 1
/usr/include/c++/4.2/bits/localefwd.h:102:34: error: macro "toupper" passed
2 arguments, but takes just 1
/usr/include/c++/4.2/bits/localefwd.h:106:34: error: macro "tolower" passed
2 arguments, but takes just 1

..........
I guess this must mean that a particular macro is in play when the
definitions from this file are used. 

localefwd.h looks like

_GLIBCXX_BEGIN_NAMESPACE(std)

  // 22.1.1 Locale
  class locale;

  // 22.1.3 Convenience interfaces
  template<typename _CharT>
    inline bool
    isspace(_CharT, const locale&);

  template<typename _CharT>
    inline bool
    isprint(_CharT, const locale&);

Original issue reported on code.google.com by replabro...@gmail.com on 28 Oct 2009 at 11:11

[GoogleCodeExporter]

This should be a defect (I used the defect template at least). Anyhow I forgot 
my
configure line which looked like this

./configure --prefix=/home/memememe/UNLADEN --with-unicode=ucs2

Original comment by replabro...@gmail.com on 28 Oct 2009 at 11:15

[GoogleCodeExporter]

This looks like http://gcc.gnu.org/bugzilla/show_bug.cgi?id=31005
What are your GCC and libstdc++ versions?

Original comment by abbeyj on 28 Oct 2009 at 12:59

[GoogleCodeExporter]

$ gcc --version
gcc (GCC) 4.2.1 20070719  [FreeBSD]
Copyright (C) 2007 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

~/devel/unladen-2009Q3:
$ g++ --version
g++ (GCC) 4.2.1 20070719  [FreeBSD]
Copyright (C) 2007 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

I have /usr/lib/libstdc++.so.6, but am not exactly sure which exact version.

Original comment by replabro...@gmail.com on 28 Oct 2009 at 2:34

[GoogleCodeExporter]

That's old enough that it could be the bug that I referenced above.  Can you 
try with
a newer version of GCC?  I'm not that familiar with FreeBSD but they seem to 
have
more recent versions of GCC available in the ports tree.  Something like:

# cd /usr/ports/lang/gcc44 && make install

And then use CC=gcc44 and CXX=g++44 to re-configure and build Unladen

Original comment by abbeyj on 28 Oct 2009 at 3:15

[GoogleCodeExporter]

I've started the gcc4.4 build and will try and report back tomorrow.  

Original comment by replabro...@gmail.com on 28 Oct 2009 at 4:35

[GoogleCodeExporter]

OK I got the gcc44 built and now the unladen-2009Q3 configure and build does go
through to the end. However, my simplistic test of working

$ unladen26 unladen-2009Q3/Lib/test/pystone.py
Pystone(1.1) time for 50000 passes = 1.82031
This machine benchmarks at 27467.8 pystones/second
~/devel:
$ python25 unladen-2009Q3/Lib/test/pystone.py
Pystone(1.1) time for 50000 passes = 0.984375
This machine benchmarks at 50793.7 pystones/second
~/devel:

makes python25 seem quite fast. Is this result expected because of the way that 
llvm
works?

Original comment by replabro...@gmail.com on 29 Oct 2009 at 1:11

[GoogleCodeExporter]

I'd say that's expected.  Unladen Swallow is mostly tuned for long-running 
server
processes that can take advantage of the JIT, so to model that in our 
benchmarks, we
warm up the benchmark before we start timing.  We haven't implemented this for
PyStone, but you could probably hack it to run twice.  The second run should be
faster than Python 2.5.

Original comment by reid.kle...@gmail.com on 29 Oct 2009 at 5:03

[GoogleCodeExporter]

I'm not surprised that we run PyStone slowly. It's listed in our Benchmarks 
page (http://code.google.com/p/unladen-swallow/wiki/Benchmarks) as a 
benchmark we explicitly do not use; it bears little relation to real-world 
idiomatic Python code, and certainly not to the kind of applications we're 
trying 
to optimize.

That said, I'm glad you got it building on FreeBSD. I'll try to apply the 
configure changes today.

Original comment by collinw on 29 Oct 2009 at 5:08

[GoogleCodeExporter]

Configure changes applied as r884.

Original comment by collinw on 30 Oct 2009 at 5:55

• Added labels: Type-Defect
• Removed labels: Type-Enhancement

[GoogleCodeExporter]

Robin, is this still a problem for you, or did r884 fix the build problems for 
you?

Original comment by collinw on 9 Dec 2009 at 6:06

• Changed state: Accepted

[GoogleCodeExporter]

I think the original errors have gone, but I'm seeing the following error when 
trying
to build the latest unladen-swallow-read-only with  FreeBSD 7.1-RELEASE (x686).

I used 

./configure --prefix=/home/rptlab/UNLADEN --with-unicode=ucs2
gmake

..........
g++ -pthread -c -IUtil/llvm/include -I./Util/llvm/include -D__STDC_LIMIT_MACROS 

-D__STDC_CONSTANT_MACROS -Woverloaded-virtual -fno-strict-aliasing -fwrapv 
-Wall    
                                                                -DNDEBUG -g -O3  -I.
-IInclude -I. -I./Include   -DPy_BUILD_CORE -Wno-write-str                      

                                         ings -o Python/llvm_fbuilder.o
Python/llvm_fbuilder.cc
In file included from Python/llvm_fbuilder.cc:24:
Util/llvm/include/llvm/Intrinsics.h:40:35: error: llvm/Intrinsics.gen: No such 
f    
                                                               ile or directory
In file included from Python/llvm_fbuilder.cc:24:
Util/llvm/include/llvm/Intrinsics.h:42: error: expected identifier before ',' 
to    
                                                               ken
Python/llvm_fbuilder.cc: In member function 'void 
py::LlvmFunctionBuilder::MemCp    
                                                               y(llvm::Value*,
llvm::Value*, llvm::Value*)':
Python/llvm_fbuilder.cc:915: error: 'memcpy' is not a member of 'Intrinsic'
gmake: *** [Python/llvm_fbuilder.o] Error 1

Original comment by replabro...@gmail.com on 10 Dec 2009 at 11:51

[GoogleCodeExporter]

Whoops I think I forgot to use gcc44. I'll try again and report back later.

Original comment by replabro...@gmail.com on 10 Dec 2009 at 11:54

[GoogleCodeExporter]

OK now after exporting the gcc44 vars I see this error after the configure 
gmake dance

g++44 -pthread -c -IUtil/llvm/include -I./Util/llvm/include 
-D__STDC_LIMIT_MACROS
-D__STDC_CONSTANT_MACROS -Woverloaded-virtual -fno-strict-aliasing              

-fwrapv -Wall -DNDEBUG -g -O3  -I. -IInclude -I. -I./Include   -DPy_BUILD_CORE
-Wno-write-strings -o Util/PyAliasAnalysis.o Util/PyAliasAnalysis.cc
In file included from /usr/local/lib/gcc44/include/c++/bits/basic_ios.h:39,
                 from /usr/local/lib/gcc44/include/c++/ios:45,
                 from /usr/local/lib/gcc44/include/c++/ostream:40,
                 from /usr/local/lib/gcc44/include/c++/iterator:65,
                 from Util/llvm/include/llvm/ADT/DenseMap.h:21,
                 from Util/llvm/include/llvm/ADT/DenseSet.h:17,
                 from ./Python/global_llvm_data.h:17,
                 from ./Util/PyTypeBuilder.h:14,
                 from ./Util/ConstantMirror.h:11,
                 from Util/PyAliasAnalysis.cc:3:
/usr/local/lib/gcc44/include/c++/bits/locale_facets.h:240:53: error: macro 
"toupper"
passed 2 arguments, but takes just 1
/usr/local/lib/gcc44/include/c++/bits/locale_facets.h:269:53: error: macro 
"tolower"
passed 2 arguments, but takes just 1
/usr/local/lib/gcc44/include/c++/bits/locale_facets.h:812:53: error: macro 
"toupper"
passed 2 arguments, but takes just 1
/usr/local/lib/gcc44/include/c++/bits/locale_facets.h:845:53: error: macro 
"tolower"
passed 2 arguments, but takes just 1
/usr/local/lib/gcc44/include/c++/bits/locale_facets.h:2521:44: error: macro 
"isspace"
passed 2 arguments, but takes just 1
/usr/local/lib/gcc44/include/c++/bits/locale_facets.h:2539:44: error: macro 
"isupper"
passed 2 arguments, but takes just 1
/usr/local/lib/gcc44/include/c++/bits/locale_facets.h:2545:44: error: macro 
"islower"
passed 2 arguments, but takes just 1
/usr/local/lib/gcc44/include/c++/bits/locale_facets.h:2551:44: error: macro 
"isalpha"
passed 2 arguments, but takes just 1
/usr/local/lib/gcc44/include/c++/bits/locale_facets.h:2575:44: error: macro 
"isalnum"
passed 2 arguments, but takes just 1
/usr/local/lib/gcc44/include/c++/bits/locale_facets.h:2587:44: error: macro 
"toupper"
passed 2 arguments, but takes just 1
/usr/local/lib/gcc44/include/c++/bits/locale_facets.h:2593:44: error: macro 
"tolower"
passed 2 arguments, but takes just 1
In file included from /usr/local/lib/gcc44/include/c++/bits/basic_ios.h:39,
                 from /usr/local/lib/gcc44/include/c++/ios:45,
                 from /usr/local/lib/gcc44/include/c++/ostream:40,
                 from /usr/local/lib/gcc44/include/c++/iterator:65,
                 from Util/llvm/include/llvm/ADT/DenseMap.h:21,
                 from Util/llvm/include/llvm/ADT/DenseSet.h:17,
                 from ./Python/global_llvm_data.h:17,
                 from ./Util/PyTypeBuilder.h:14,
                 from ./Util/ConstantMirror.h:11,
                 from Util/PyAliasAnalysis.cc:3:
/usr/local/lib/gcc44/include/c++/bits/locale_facets.h:225: error: 'btowc' is 
not a type
/usr/local/lib/gcc44/include/c++/bits/locale_facets.h:240: error: expected ';' 
before
'const'
/usr/local/lib/gcc44/include/c++/bits/locale_facets.h:253: error: expected ';' 
before
'char_type'
/usr/local/lib/gcc44/include/c++/bits/locale_facets.h:254: error: 'btowc' is 
not a type
/usr/local/lib/gcc44/include/c++/bits/locale_facets.h:269: error: expected ';' 
before
'const'
/usr/local/lib/gcc44/include/c++/bits/locale_facets.h:285: error: expected ';' 
before
'char_type'
/usr/local/lib/gcc44/include/c++/bits/locale_facets.h: In member function 
'_CharT
std::__ctype_abstract_base<_CharT>::towupper(int (*)(_CharT)) cons              
   t':
/usr/local/lib/gcc44/include/c++/bits/locale_facets.h:226: error: '__c' was not
declared in this scope
/usr/local/lib/gcc44/include/c++/bits/locale_facets.h: In member function 
'_CharT
std::__ctype_abstract_base<_CharT>::towlower(int (*)(_CharT)) cons              
   t':
/usr/local/lib/gcc44/include/c++/bits/locale_facets.h:255: error: '__c' was not
declared in this scope
/usr/local/lib/gcc44/include/c++/bits/locale_facets.h: At global scope:
/usr/local/lib/gcc44/include/c++/bits/locale_facets.h:795: error: 'btowc' is 
not a type
/usr/local/lib/gcc44/include/c++/bits/locale_facets.h:812: error: expected ';' 
before
'const'
/usr/local/lib/gcc44/include/c++/bits/locale_facets.h:827: error: expected ';' 
before
'char_type'
/usr/local/lib/gcc44/include/c++/bits/locale_facets.h:828: error: 'btowc' is 
not a type
/usr/local/lib/gcc44/include/c++/bits/locale_facets.h:845: error: expected ';' 
before
'const'
/usr/local/lib/gcc44/include/c++/bits/locale_facets.h:864: error: expected ';' 
before
'char_type'
/usr/local/lib/gcc44/include/c++/bits/locale_facets.h: In member function 'char
std::ctype<char>::towupper(int (*)(char)) const':
/usr/local/lib/gcc44/include/c++/bits/locale_facets.h:796: error: '__c' was not
declared in this scope
/usr/local/lib/gcc44/include/c++/bits/locale_facets.h: In member function 'char
std::ctype<char>::towlower(int (*)(char)) const':
/usr/local/lib/gcc44/include/c++/bits/locale_facets.h:829: error: '__c' was not
declared in this scope
In file included from /usr/local/lib/gcc44/include/c++/bits/basic_ios.h:39,
                 from /usr/local/lib/gcc44/include/c++/ios:45,
                 from /usr/local/lib/gcc44/include/c++/ostream:40,
                 from /usr/local/lib/gcc44/include/c++/iterator:65,
                 from Util/llvm/include/llvm/ADT/DenseMap.h:21,
                 from Util/llvm/include/llvm/ADT/DenseSet.h:17,
                 from ./Python/global_llvm_data.h:17,
                 from ./Util/PyTypeBuilder.h:14,
                 from ./Util/ConstantMirror.h:11,
                 from Util/PyAliasAnalysis.cc:3:
/usr/local/lib/gcc44/include/c++/bits/locale_facets.h: At global scope:
/usr/local/lib/gcc44/include/c++/bits/locale_facets.h:2521: error: 
'std::isspace'
declared as an 'inline' variable
/usr/local/lib/gcc44/include/c++/bits/locale_facets.h:2521: error: template
declaration of 'bool std::isspace'
/usr/local/lib/gcc44/include/c++/bits/locale_facets.h:2522: error: expected
primary-expression before 'return'
/usr/local/lib/gcc44/include/c++/bits/locale_facets.h:2522: error: expected '}'
before 'return'
/usr/local/lib/gcc44/include/c++/bits/locale_facets.h:2527: error: expected ',' 
or
'...' before '&' token
/usr/local/lib/gcc44/include/c++/bits/locale_facets.h: In function 'bool
isprint(_CharT, int)':
/usr/local/lib/gcc44/include/c++/bits/locale_facets.h:2528: error: 'use_facet' 
was
not declared in this scope
/usr/local/lib/gcc44/include/c++/bits/locale_facets.h:2528: error: 'ctype' was 
not
declared in this scope
/usr/local/lib/gcc44/include/c++/bits/locale_facets.h:2528: error: expected
primary-expression before '>' token
/usr/local/lib/gcc44/include/c++/bits/locale_facets.h:2528: error: expected
primary-expression before '>' token
/usr/local/lib/gcc44/include/c++/bits/locale_facets.h:2528: error: '__loc' was 
not
declared in this scope
/usr/local/lib/gcc44/include/c++/bits/locale_facets.h:2528: error: 'ctype_base' 
has
not been declared
/usr/local/lib/gcc44/include/c++/bits/locale_facets.h: At global scope:
/usr/local/lib/gcc44/include/c++/bits/locale_facets.h:2533: error: expected ',' 
or
'...' before '&' token
/usr/local/lib/gcc44/include/c++/bits/locale_facets.h: In function 'bool
iscntrl(_CharT, int)':
/usr/local/lib/gcc44/include/c++/bits/locale_facets.h:2534: error: 'use_facet' 
was
not declared in this scope
/usr/local/lib/gcc44/include/c++/bits/locale_facets.h:2534: error: 'ctype' was 
not
declared in this scope
/usr/local/lib/gcc44/include/c++/bits/locale_facets.h:2534: error: expected
primary-expression before '>' token
/usr/local/lib/gcc44/include/c++/bits/locale_facets.h:2534: error: expected
primary-expression before '>' token
/usr/local/lib/gcc44/include/c++/bits/locale_facets.h:2534: error: '__loc' was 
not
declared in this scope
/usr/local/lib/gcc44/include/c++/bits/locale_facets.h:2534: error: 'ctype_base' 
has
not been declared
/usr/local/lib/gcc44/include/c++/bits/locale_facets.h: At global scope:
/usr/local/lib/gcc44/include/c++/bits/locale_facets.h:2539: error: 'isupper' 
declared
as an 'inline' variable
/usr/local/lib/gcc44/include/c++/bits/locale_facets.h:2539: error: template
declaration of 'bool isupper'
/usr/local/lib/gcc44/include/c++/bits/locale_facets.h:2540: error: expected
primary-expression before 'return'
/usr/local/lib/gcc44/include/c++/bits/locale_facets.h:2540: error: expected '}'
before 'return'
/usr/local/lib/gcc44/include/c++/bits/locale_facets.h:2540: error: expected
declaration before '}' token
gmake: *** [Util/PyAliasAnalysis.o] Error 1
~/devel/unladen-swallow-read-only:

Original comment by replabro...@gmail.com on 10 Dec 2009 at 12:03

[GoogleCodeExporter]

This is strange.  The locale_facets.h file is supposed to #include <cctype>.  
That
would #undef these macros that are causing you trouble.  Can you look in
/usr/local/lib/gcc44/include/c++/bits/locale_facets.h and see if there is an 
#include
<cctype> line near the top?

If so, can you find out which cctype file your compiler is using?  Something 
like
echo "#include <cctype>" | cpp44 -xc++ -v 2> /dev/null | grep cctype
should tell you.  Then look in that file and see if there's an "#undef toupper" 
line
in it.

If there is then perhaps something other than <ctype.h> is providing a 
definition for
the toupper macro.  Try searching all files under /usr/include for "define 
toupper"
and see if you get any hits.

Original comment by abbeyj on 14 Dec 2009 at 3:30

[GoogleCodeExporter]

1) There is an include of  <cctype> in
/usr/local/lib/gcc44/include/c++/bits/locale_facets.h it begins

#ifndef _LOCALE_FACETS_H
#define _LOCALE_FACETS_H 1

#pragma GCC system_header

#include <cwctype>  // For wctype_t
#include <cctype>

2) 

$ echo "#include <cctype>" | cpp44 -xc++ -v 2> /dev/null | grep cctype
# 1 "/usr/local/lib/gcc44/include/c++/cctype" 1 3
.........
# 64 "/usr/local/lib/gcc44/include/c++/cctype" 3

in /usr/local/lib/gcc44/include/c++/cctype I see the following

#pragma GCC system_header

#include <bits/c++config.h>
#include <ctype.h>

#ifndef _GLIBCXX_CCTYPE
#define _GLIBCXX_CCTYPE 1

// Get rid of those macros defined in <ctype.h> in lieu of real functions.
#undef isalnum
.......
#undef toupper

_GLIBCXX_BEGIN_NAMESPACE(std)

  using ::isalnum;
  using ::isalpha;
  using ::iscntrl;

3) I tried searching and find these other 'define toupper' lines
find . -type f | xargs -J % grep 'define toupper' %
./netinet/ip_proxy.h:#define toupper(x) (isupper(x) ? (x) : (x) - 'a' + 'A')
./sys/ctype.h:#define toupper(c)        ((c) - 0x20 * (((c) >= 'a') && ((c) <= 
'z')))
/usr/include:
$ find /usr/local/include -type f | xargs -J % grep 'define toupper' %
/usr/local/include/python2.5/pyport.h:#define toupper(c) towupper(btowc(c))

Original comment by replabro...@gmail.com on 14 Dec 2009 at 10:08

[GoogleCodeExporter]

I think I understand the problem now.  PyAliasAnalysis.cc #includes
Util/PyAliasAnalysis.h first which #includes llvm/Pass.h.  And that must somehow
eventually #include cctype which #includes ctype.h and then #undefs the macros.
Then PyAliasAnalysis.cc #includes Util/ConstantMirror.h which #includes Python.h
which #includes pyport.h which, on FreeBSD only, #includes ctype.h and then 
#undefs
and provides its own definitions for the macros.  Then later we indirectly 
#include
locale_facets.h which expects the #include cctype to #undef all the macros it 
doesn't
want.  But since cctype has already been #included once, the header guard means 
that
it does nothing.  The macros stay defined and we run into the problem.

So, one solution here is to #include Python.h at the top of all the .cc files 
so it
will always be #included before cctype.
Or we can change pyport.h to not always redefine the macros (either by using 
#ifdef
toupper, etc. or #ifdef __cplusplus).

Collin, what do you think?

Original comment by abbeyj on 14 Dec 2009 at 2:35

[GoogleCodeExporter]

I think the right thing to do is change pyport.h to conditionally define the 
macros.

Original comment by collinw on 14 Dec 2009 at 6:27

[GoogleCodeExporter]

Can you see if the patch at http://codereview.appspot.com/179049 fixes this 
problem
for you?

Original comment by abbeyj on 15 Dec 2009 at 12:53

[GoogleCodeExporter]

I checked out a fresh copy of readonly and applied the patch. 

Then did the setup 

export CC=gcc44 CXX=g++44
./configure --prefix=/home/rptlab/UNLADEN --with-unicode=ucs2
gmake

The original error is certainly gone, but now (after an age of compiling 
llvm/clang
etc etc) I'm getting an ld error (see below). If it is of interest I do have the
script output of the gmake.

llvm[2]: Linking Release+Debug-Asserts executable llvm-dis
llvm[2]: ======= Finished Linking Release+Debug-Asserts Executable llvm-dis
gmake[2]: Leaving directory
`/usr/home/rptlab/devel/unladen-swallow-read-only/Util/llvm/too
gmake[1]: Leaving directory
`/usr/home/rptlab/devel/unladen-swallow-read-only/Util/llvm/too
Util/llvm/Release+Debug-Asserts/bin/clang -O3 -emit-llvm -c -I. -IInclude -I. 
-I./Include
Util/llvm/Release+Debug-Asserts/bin/llvm-link -o Modules/libpython2.6.unopt.bc
Python/llvm_
Util/llvm/Release+Debug-Asserts/bin/opt -o Modules/libpython2.6.bc -O3
Modules/libpython2.6
g++44 -pthread  -o python Modules/python.o \
        -Wl,--export-dynamic libpython2.6.a -lutil   -lm
libpython2.6.a(posixmodule.o)(.text+0x686): In function `posix_tmpnam':
./Modules/posixmodule.c:7081: warning: warning: tmpnam() possibly used unsafely;
consider u
libpython2.6.a(posixmodule.o)(.text+0x765): In function `posix_tempnam':
./Modules/posixmodule.c:7034: warning: warning: tempnam() possibly used 
unsafely;
consider
libpython2.6.a(global_llvm_data.o)(.text+0x9b): In function `_PyLlvm_Init':
Util/llvm/include/llvm/ADT/SmallVector.h:449: undefined reference to
`llvm::cl::ParseEnviro
libpython2.6.a(global_llvm_data.o)(.text+0xf5): In function
`PyGlobalLlvmData::Optimize(llv
Util/llvm/include/llvm/ADT/SmallVector.h:466: undefined reference to
`llvm::FunctionPassMan
libpython2.6.a(global_llvm_data.o)(.text+0x1f5): In function
`PyGlobalLlvmData::MaybeCollec
Python/global_llvm_data.cc:356: undefined reference to
`llvm::PassManager::run(llvm::Module
libpython2.6.a(global_llvm_data.o)(.text+0x2bf): In function
`PyGlobalLlvmData::GetGlobalSt
Util/llvm/include/llvm/ADT/StringMap.h:355: undefined reference to
`llvm::StringMapImpl::Lo
libpython2.6.a(global_llvm_data.o)(.text+0x303): In function
`PyGlobalLlvmData::GetGlobalSt
Util/llvm/include/llvm/Support/ValueHandle.h:74: undefined reference to
`llvm::ValueHandleB
libpython2.6.a(global_llvm_data.o)(.text+0x314): In function
`PyGlobalLlvmData::GetGlobalSt
./Python/global_llvm_data.h:56: undefined reference to 
`llvm::getGlobalContext()'
libpython2.6.a(global_llvm_data.o)(.text+0x31c): In function
`PyGlobalLlvmData::GetGlobalSt
...................
...................
libpython2.6.a(llvm_fbuilder.o)(.gnu.linkonce.t._ZN2py19LlvmFunctionBuilder17Get
GlobalFunctionIFvP7_objectS3_S3_EEEPN4llvm8FunctionERKSs+0xeb):
In function `llvm::Function* py::LlvmFunctionBuilder::GetGlobalFunction<void
()(_object*, _object*, _object*)>(std::basic_string<char, 
std::char_traits<char>,
std::allocator<char> > const&)':
Util/llvm/include/llvm/Support/TypeBuilder.h:260: undefined reference to
`llvm::FunctionType::get(llvm::Type const*, std::vector<llvm::Type const*,
std::allocator<llvm::Type const*> > const&, bool)'
libpython2.6.a(llvm_fbuilder.o)(.gnu.linkonce.t._ZN2py19LlvmFunctionBuilder17Get
GlobalFunctionIFvP7_objectS3_S3_EEEPN4llvm8FunctionERKSs+0x11a):
In function `llvm::Function* py::LlvmFunctionBuilder::GetGlobalFunction<void
()(_object*, _object*, _object*)>(std::basic_string<char, 
std::char_traits<char>,
std::allocator<char> > const&)':
Util/llvm/include/llvm/Support/Casting.h:225: undefined reference to
`llvm::Module::getOrInsertFunction(llvm::StringRef, llvm::FunctionType const*)'
collect2: ld returned 1 exit status
gmake: *** [python] Error 1

Original comment by rptl...@gmail.com on 15 Dec 2009 at 11:04

[GoogleCodeExporter]

That kind of thing could happen when libpython.a got built before some of the 
LLVM 
libraries it depended on. You would see an error from `ar` toward the top of 
the build 
log, but because of a bug in our Makefile it wouldn't stop the build. I fixed 
that bug and 
got the ordering (closer to) right in r938. Sorry for the extra work, but could 
you sync 
and try again? You probably won't have to `make clean`.

Thanks for the help.

Original comment by jyass...@gmail.com on 17 Dec 2009 at 10:00

[GoogleCodeExporter]

OK I did the update and then another gmake; things certainly proceed further 
now.

Right at the end of the build I notice these messages; are these significant?

Failed to find the necessary bits to build these modules:
_sqlite3           _tkinter           bsddb185
dl                 gdbm               imageop
linuxaudiodev      spwd               sunaudiodev
To find the necessary bits, look in setup.py in detect_modules() for the 
module's name.

Failed to build these modules:
_locale

running build_scripts
creating build/scripts-2.6
copying and adjusting
/usr/home/rptlab/devel/unladen-swallow-read-only/Tools/scripts/pydoc -> 
build/scripts-2.6
copying and adjusting
/usr/home/rptlab/devel/unladen-swallow-read-only/Tools/scripts/idle -> 
build/scripts-2.6
copying and adjusting
/usr/home/rptlab/devel/unladen-swallow-read-only/Tools/scripts/2to3 -> 
build/scripts-2.6
copying and adjusting 
/usr/home/rptlab/devel/unladen-swallow-read-only/Lib/smtpd.py
-> build/scripts-2.6
changing mode of build/scripts-2.6/pydoc from 644 to 755
changing mode of build/scripts-2.6/idle from 644 to 755
changing mode of build/scripts-2.6/2to3 from 644 to 755
changing mode of build/scripts-2.6/smtpd.py from 644 to 755

Original comment by rptl...@gmail.com on 18 Dec 2009 at 10:19

[GoogleCodeExporter]

"Failed to find" messages usually mean your system doesn't have some library 
that 
module depends on. They're generally harmless. "Failed to build" messages are 
usually accompanied by some sort of compiler or linker error. For example, when 
I 
compile Python on OSX with MacPorts gcc-4.4, I get:

building '_locale' extension
gcc -fno-strict-aliasing -fwrapv -Wall -Wstrict-prototypes -g -I. -
I/Users/jyasskin/src/unladen-swallow/trunk/src/./Include -
I/Users/jyasskin/src/unladen-swallow/trunk/src/./Mac/Include -I. -IInclude 
-I../src 
-I/usr/local/include -I../src/Include -I/Users/jyasskin/src/unladen-
swallow/trunk/dbg -c /Users/jyasskin/src/unladen-
swallow/trunk/src/Modules/_localemodule.c -o build/temp.macosx-10.3-i386-
2.6-pydebug/Users/jyasskin/src/unladen-
swallow/trunk/src/Modules/_localemodule.o
gcc -bundle -undefined dynamic_lookup build/temp.macosx-10.3-i386-2.6-
pydebug/Users/jyasskin/src/unladen-swallow/trunk/src/Modules/_localemodule.o -
L/usr/local/lib -ldl -o build/lib.macosx-10.3-i386-2.6-pydebug/_locale.so -
framework CoreFoundation
*** WARNING: renaming "_locale" since importing it failed: 
dlopen(build/lib.macosx-
10.3-i386-2.6-pydebug/_locale.so, 2): Symbol not found: _libintl_bindtextdomain
  Referenced from: /Users/jyasskin/src/unladen-
swallow/trunk/dbg/build/lib.macosx-10.3-i386-2.6-pydebug/_locale.so
  Expected in: dynamic lookup

(You only get the gcc lines if you use plain `make` without -j.) Mine is 
probably 
caused by a missing #ifdef in _localemodule.c. You may have a different error.

Missing _locale may cause a couple test failures, but most of the test suite 
should 
still pass, and Python should work unless you try to import that module.  How 
does 
the test suite do on your box?

Original comment by jyass...@gmail.com on 18 Dec 2009 at 6:39

[GoogleCodeExporter]

I get these outputs from testall.py. I also had to interrupt a threading test 
which
seemed to hang.

test test_smtplib failed -- Traceback (most recent call last):
  File "/home/rptlab/UNLADEN/lib/python2.6/test/test_smtplib.py", line 225, in testSend
    self.assertEqual(self.output.getvalue(), mexpect)
AssertionError: "---------- MESSAGE FOLLOWS ----------\nA test 
message\n------------
END MESSAGE ------------\nerror: uncaptured python exception, closing channel
<smtpd.SMTPChannel 127.0.0.1:61298 at 0x80c62e560> (<class 
'socket.error'>:[Errno 9]
Bad file descriptor 
[/home/rptlab/UNLADEN/lib/python2.6/asyncore.py|readwrite|101]
[/home/rptlab/UNLADEN/lib/python2.6/asyncore.py|handle_write_event|427]
[<string>|getsockopt|1] 
[/home/rptlab/UNLADEN/lib/python2.6/socket.py|_dummy|165])\n"
!= '---------- MESSAGE FOLLOWS ----------\nA test message\n------------ END 
MESSAGE
------------\n'

I got these after patching and restarting the build. I will repeat after a gmake
clean in case that has something to do with failure to build the _locale module.

Original comment by replabro...@gmail.com on 19 Dec 2009 at 1:04

[GoogleCodeExporter]

test_smtplib is known to be flaky, so I wouldn't worry about that one. Does the 
threading test hang every time? Which threading test is it?

Original comment by collinw on 19 Dec 2009 at 9:42

[GoogleCodeExporter]

The _locale problem appears to be because when built it fails to import ie I 
see this
in the gmake

gcc44 -shared build/temp.freebsd-7.1-RELEASE-amd64-2.6/usr/home/rptlab/devel/unl
building '_locale' extension
gcc44 -pthread -fPIC -fno-strict-aliasing -fwrapv -Wall -Wstrict-prototypes -DND
gcc44 -shared build/temp.freebsd-7.1-RELEASE-amd64-2.6/usr/home/rptlab/devel/unl
*** WARNING: renaming "_locale" since importing it failed: build/lib.freebsd-7.1

I copied _locale_failed.so into the dynload folder and then
import _local_failed produces this

$ unladen26
Python 2.6.1 (r261:942M, Dec 19 2009, 13:52:00)
[GCC 4.4.3 20091020 (prerelease)] on freebsd7
[Unladen Swallow 2009Q4]
Type "help", "copyright", "credits" or "license" for more information.
>>> import _locale_failed
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: /home/rptlab/UNLADEN/lib/python2.6/lib-dynload/_locale_failed.so:
Undefined symbol "libintl_bindtextdomain"
>>>

During testall.py it appears to hang in test_threading

the output looks like
test_threading
test_PyThreadState_SetAsyncExc (test.test_threading.ThreadTests) ...     started
worker thread
    trying nonsensical thread id
    waiting for worker thread to get started
    verifying worker hasn't exited
    attempting to raise asynch exception in worker
    waiting for worker to say it caught the exception
    all OK -- joining worker
ok
test_enumerate_after_join (test.test_threading.ThreadTests) ... ok
test_finalize_runnning_thread (test.test_threading.ThreadTests) ... ok
test_finalize_with_trace (test.test_threading.ThreadTests) ... ok
test_foreign_thread (test.test_threading.ThreadTests) ... ok
test_no_refcycle_through_target (test.test_threading.ThreadTests) ... ok
test_various_ops (test.test_threading.ThreadTests) ... task <thread 0> will run 
for
60.9 usec
 task <thread 1> will run for 15.5 usec
1 tasks are running
task <thread 2> will run for 85.7 usec
 task <thread 3> will run for 74.0 usec
2 tasks are running
 task <thread 4> will run for 98.4 usec
task <thread 1> 3 tasks are runningtask <thread 6> will run for 76.9 usec

done
 task <thread 0> done
<thread 1> is finished. 2 tasks are running
task <thread 5> will run for 82.8 usec
 <thread 0> is finished. 1 tasks are running
task <thread 7> will run for 52.0 usec
 2 tasks are running
task <thread 3>task <thread 8> will run for 12.4 usec
 done
 3task <thread 9> will run for 90.0 usec
 waiting for all tasks to complete
 tasks are running
task <thread 2> done
<thread 3> is finished. 2 tasks are running
<thread 2> is finished. 1 tasks are running
2 tasks are running
3 tasks are running
task <thread 4> done
<thread 4> is finished. 2 tasks are running
3 tasks are running
task <thread 6> done
<thread 6> is finished. 2 tasks are running
 3 tasks are running
task <thread 5> done
<thread 5> is finished. 2 tasks are running
3 tasks are running
task <thread 7> done
<thread 7> is finished. 2 tasks are running
task <thread 8> task <thread 9> done
done
<thread 8> is finished. 1 tasks are running
<thread 9> is finished. 0 tasks are running
all tasks done
ok
test_various_ops_large_stack (test.test_threading.ThreadTests) ... with 1MB 
thread
stack size...
task <thread 0> will run for 90.4 usec
 task <thread 1> will run for 61.5 usec
1 task <thread 2> will run for 10.8 usec
tasks are running
2 tasks are runningtask <thread 3> will run for 54.8 usec

 task <thread 4> will run for 80.3 usec
3 tasks are running
 task <thread 5> will run for 99.8 usec
task <thread 6> will run for 34.3 usec
 task <thread 0> task <thread 7> will run for 12.4 usec
done
<thread 0> is finished. 2 tasks are running
3 tasks are running
task <thread 8> will run for 1.5 usec
 task <thread 2> done
task <thread 1> task <thread 9> will run for 22.6 usec
done
 <thread 2> is finished. 2 tasks are running
waiting for all tasks to complete
task <thread 7> done
 <thread 1> is finished. 1 tasks are running
2 tasks are running
3 tasks are running
<thread 7> is finished. 2 tasks are running
3 tasks are running
task <thread 4> task <thread 5> done
<thread 5> is finished. 2 tasks are running
done
3 tasks are running
<thread 4> is finished. 2 tasks are running
3 tasks are running
task <thread 6> done
<thread 6> is finished. 2 tasks are running
3 tasks are running
task <thread 8> done
 <thread 8> is finished. 2 tasks are running
task <thread 3> done
<thread 3> is finished. 1 tasks are running
task <thread 9> done
<thread 9> is finished. 0 tasks are running
all tasks done
ok
test_various_ops_small_stack (test.test_threading.ThreadTests) ... with 256kB 
thread
stack size...
task <thread 0> will run for 89.5 usec
 task <thread 1> will run for 97.8 usec
1 tasks are running
 task <thread 2> will run for 56.9 usec
 task <thread 3> will run for 3.4 usec
2 tasks are runningtask <thread 4> will run for 49.2 usec

3 tasks are running
 task <thread 5> will run for 57.9 usec
task <thread 6> will run for 39.1 usec
 task <thread 7> will run for 62.6 usec
task <thread 0> done
 task <thread 0> is finished. 2 tasks are running
<thread 1> done
 task <thread 8> will run for 26.7 usec
<thread 1> is finished. 1 tasks are running
waiting for all tasks to complete
task <thread 3>2 tasks are running
 done
task <thread 9> will run for 65.1 usec
3 tasks are running
<thread 3> is finished. 2 tasks are running
3 tasks are running
task <thread 8> done
 task <thread 9> done
 <thread 8> is finished. 2 tasks are running
<thread 9> is finished. 1 tasks are running
2 tasks are running
3 tasks are running
task <thread 5> done
<thread 5> is finished. 2 tasks are running
3 tasks are running
task <thread 6> done
<thread 6> is finished. 2 tasks are running
3 tasks are running
task <thread 7> donetask <thread 2> done

<thread 2> is finished. 2 tasks are running
<thread 7> is finished. 1 tasks are running
task <thread 4> done
<thread 4> is finished. 0 tasks are running
all tasks done
ok
test_1_join_on_shutdown (test.test_threading.ThreadJoinOnShutdown) ... ok
test_2_join_in_forked_process (test.test_threading.ThreadJoinOnShutdown) ... ok
test_3_join_in_forked_from_thread (test.test_threading.ThreadJoinOnShutdown) 
... ^C
275 tests OK.
2 tests failed:
    test_io test_locale
36 tests skipped:
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/home/rptlab/UNLADEN/lib/python2.6/threading.py", line 537, in __bootstrap_inner
    self.run()
  File "/home/rptlab/UNLADEN/lib/python2.6/threading.py", line 492, in run
    self.__target(*self.__args, **self.__kwargs)
  File "<string>", line 14, in worker
OSError: [Errno 4] Interrupted system call

Exception KeyboardInterrupt in <module 'threading' from
'/home/rptlab/UNLADEN/lib/python2.6/threading.pyc'> ignored
    test__locale test_aepack test_al test_applesingle test_bsddb185
    test_bsddb3 test_cd test_cl test_codecmaps_cn test_codecmaps_hk
    test_codecmaps_jp test_codecmaps_kr test_codecmaps_tw test_curses
    test_dl test_epoll test_gdbm test_gl test_imageop test_imgfile
    test_jit_gdb test_lib2to3 test_linuxaudiodev test_macos
    test_macostools test_multiprocessing test_normalization
    test_ossaudiodev test_pep277 test_py3kwarn test_scriptpackages
    test_socketserver test_sqlite test_startfile test_sunaudiodev
    test_tcl
4 skips unexpected on freebsd7:
    test__locale test_lib2to3 test_dl test_bsddb185
~/UNLADEN/lib/python2.6/test:
$ exit

Script done, output file is /tmp/ttt
~/UNLADEN/lib/python2.6/test:
$

Original comment by rptl...@gmail.com on 21 Dec 2009 at 11:33

[GoogleCodeExporter]

Not sure if this is relevant and it's something I have completely missed 
building on
this machine is that this freebsd is actually a 64 bit version

ie uname -a

FreeBSD app3.reportlab.com 7.1-RELEASE FreeBSD 7.1-RELEASE #0: Thu Jan  1 
08:58:24
UTC 2009     root@driscoll.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC  amd64

Original comment by rptl...@gmail.com on 21 Dec 2009 at 11:36

[GoogleCodeExporter]

I have tried rebuilding on a 32bit x386 FreeBSD 8.0 machine with gcc44 and get 
much
the same outcome ie  module _locale fails to import (for the same reason). 
However, I
don't get any hangs in test_threading so the amd64 has something to do with 
that. My
freebsd x386 test_all.py ends with

OK
322 tests OK.
3 tests failed:
    test_io test_locale test_smtplib
39 tests skipped:
    test__locale test_aepack test_al test_applesingle test_bsddb185
    test_bsddb3 test_cd test_cl test_codecmaps_cn test_codecmaps_hk
    test_codecmaps_jp test_codecmaps_kr test_codecmaps_tw test_curses
    test_epoll test_gl test_imgfile test_jit_gdb test_lib2to3
    test_linuxaudiodev test_macos test_macostools test_multiprocessing
    test_normalization test_ossaudiodev test_pep277 test_py3kwarn
    test_scriptpackages test_socketserver test_startfile
    test_sunaudiodev test_tcl test_timeout test_unicode_file
    test_urllib2net test_urllibnet test_winreg test_winsound
    test_zipfile64
3 skips unexpected on freebsd8:
    test__locale test_lib2to3 test_bsddb185

Original comment by replabro...@gmail.com on 22 Dec 2009 at 5:38

[GoogleCodeExporter]

Did a bit of checking with the x386 FreeBSD-8.0 system. Python-2.6 builds 
_locale.so
OK and doesn't seem to be using libintl.

pyconfig.h:/* #undef HAVE_LIBINTL_H */
pyconfig.h:/* #undef WITH_LIBINTL */

also no references in the .so file.

The unladen build of _locale does seem to produce dyn so which does have 
references
to libintl, but for whatever reason the unladen pyconfig doesn't define 
WITH_LIBINTL
either the unladen pyconfig contains 
#define HAVE_LIBINTL_H 1
/* #undef WITH_LIBINTL */

looking in the ouput of the configure shows

checking for textdomain in -lintl... no

so perhaps the test is failing ie do I need more than 

export CC=gcc44 CXX=g++44

to make the configure work?

Original comment by replabro...@gmail.com on 23 Dec 2009 at 11:57