search

OpenELEC / OpenELEC.tv

OpenELEC - The living room PC for everyone
http://openelec.tv
1.61k stars 884 forks source link

openelec/head cross-build on opensuse/64 fails @ python "AttributeError: 'module' object has no attribute 'lib'" #2964

Closed ghost closed 10 years ago

ghost commented 10 years ago

on

lsb_release -sd
    "openSUSE 12.3 (x86_64)"

cross build of openelec head

cd /usr/local/src
git clone git://github.com/OpenELEC/OpenELEC.tv.git
ln -sf /usr/local/src/OpenELEC.tv /home/rand/workspace/openelec
cd  /home/rand/workspace/openelec
git log | head
    commit 8b0cd8daf38588935765726413b6c126fe1bf84a
    Author: Stephan Raue <stephan@openelec.tv>
    Date:   Wed Mar 5 01:31:10 2014 +0100

        xbmc: add upstream patch

        Signed-off-by: Stephan Raue <stephan@openelec.tv>

    commit 7f30143204ff0bc925ed92c96c94fcee4e2b30dc
    Author: Stephan Raue <stephan@openelec.tv>

with

PROJECT=RPi ARCH=arm nice -n 2 ionice -c 3 make

fails @ python AttributeError

...
if test -d /home/rand/workspace/openelec/build.OpenELEC-RPi.arm-devel/toolchain/lib/python2.7/distutils/tests; then \
        /usr/bin/install -c -m 644 /home/rand/workspace/openelec/build.OpenELEC-RPi.arm-devel/Python-2.7.3/Modules/xxmodule.c \
                /home/rand/workspace/openelec/build.OpenELEC-RPi.arm-devel/toolchain/lib/python2.7/distutils/tests ; \
fi
PYTHONPATH=/home/rand/workspace/openelec/build.OpenELEC-RPi.arm-devel/toolchain/lib/python2.7 LD_LIBRARY_PATH=/home/rand/workspace/openelec/build.OpenELEC-RPi.arm-devel/Python-2.7.3/.x86_64-suse-linux:/home/rand/workspace/openelec/build.OpenELEC-RPi.arm-devel/toolchain/lib: \
        ./python -Wi -t -c "import lib2to3.pygram, lib2to3.patcomp;lib2to3.patcomp.PatternCompiler()"
make[1]: Leaving directory `/usr/local/src/openelec/OpenELEC.tv/build.OpenELEC-RPi.arm-devel/Python-2.7.3/.x86_64-suse-linux'
Traceback (most recent call last):
  File "/usr/lib64/python2.7/site.py", line 568, in <module>
    main()
  File "/usr/lib64/python2.7/site.py", line 550, in main
    known_paths = addusersitepackages(known_paths)
  File "/usr/lib64/python2.7/site.py", line 278, in addusersitepackages
    user_site = getusersitepackages()
  File "/usr/lib64/python2.7/site.py", line 253, in getusersitepackages
    user_base = getuserbase() # this will also set USER_BASE
  File "/usr/lib64/python2.7/site.py", line 242, in getuserbase
    from sysconfig import get_config_var
  File "/usr/lib64/python2.7/sysconfig.py", line 10, in <module>
    'stdlib': '{base}/'+sys.lib+'/python{py_version_short}',
AttributeError: 'module' object has no attribute 'lib'
make: *** [system] Error 1
sraue commented 10 years ago

hmm maybe a problem because of the symlink? can you try using sudo mount --bind /usr/local/src/openelec/OpenELEC.tv /home/rand/workspace/openelec instead?

ghost commented 10 years ago

changing the link -> mount, and trying to continue the make doesn't do it

sudo mount --bind /usr/local/src/openelec/OpenELEC.tv /home/rand/workspace/openelec
cd /home/rand/workspace/openelec

PROJECT=RPi ARCH=arm nice -n 2 ionice -c 3 make
./scripts/image

 =================================================================================
 Configuration for OpenELEC (unofficial)
 =================================================================================

 Buildoptions:
 ======================================================
 - CPU (ARCH):                           arm1176jzf-s (arm)
 - FLOAT:                                hard
 - FPU:                                  vfp
 - SIMD support:                         no
 - Optimizations:                        size
 - LTO (Link Time Optimization) support: yes
 - GOLD (Google Linker) Support:         yes
 - LLVM support:                         

 Graphic configuration:
 ======================================================
 - XORG support:                         
 - XORG Composite support:               
 - XORG Xinerama support:                
 - SDL support:                          no
 - OpenGL (GLX) support (provider):      no (no)
 - OpenGLES support (provider):          yes (bcm2835-driver)
 - WindowManager:                        none
 - Xorg Graphic Drivers:                 

 Hardware decoder configuration:
 ======================================================
 - Broadcom CrystalHD Decoder:           no
 - XBMC Player driver:                   bcm2835-driver
 - VAAPI Support:                        no
 - VDPAU Support:                        no

 Input device configuration:
 ======================================================
 - Remote support:                       yes
 - ATV Remote support:                   no
 - CEC Adapter support:                  yes
 - IRTrans support:                      no
 - XBMC Joystick support:                no

 Misc. hardware configuration:
 ======================================================
 - ALSA support:                         no
 - Pulseaudio support:                   no
 - Blu-Ray support:                      yes
 - Bluetooth support:                    yes
 - Hardware Sensors support:             yes
 - LCD drivers:                          irtrans,imon,imonlcd,mdm166a,MtxOrb,lis,dm140,hd44780,CFontz,SureElec,vlsys_m428
 - Include driver:                       RTL8192CU
 - Include driver:                       RTL8188EU
 - Include driver:                       dvbhdhomerun
 - Include firmware:                     misc-firmware
 - Include firmware:                     wlan-firmware
 - Include firmware:                     dvb-firmware

 Network service configuration:
 ======================================================
 - Avahi (Zeroconf) support:             yes
 - SAMBA server support:                 yes
 - SFTP server support:                  yes
 - PPTP support:                         yes
 - OpenVPN support:                      yes
 - XBMC Airplay support:                 yes
 - XBMC Airtunes support:                yes
 - XBMC AFP support:                     yes
 - XBMC NFS support:                     yes
 - XBMC SAMBA client support:            yes
 - XBMC Webserver support:               yes

 OS configuration:
 ======================================================
 - OEM Support:                          no
 - Default ROOT Password:                openelec
 - Bootloader:                           bcm2835-bootloader
 - UDevil support:                       yes
 - Installer support:                    no

 Misc. Filesystems:
 ======================================================
 - Swap Support:                         yes
 - Swapfile default size:                128
 - exFAT Support (via Fuse):             yes
 - NTFS Support (via Fuse):              yes
 - Install HFS Tools:                    yes

 XBMC configuration:
 ======================================================
 - XBMC version:                         xbmc
 - XBMC nonfree support:                 yes
 - XBMC DVDCSS support:                  yes
 - Include Skin:                         Confluence
 - Default Skin:                         Confluence
 - Include extra fonts:                  yes
 - Include RSXS Screensaver:             no
 - Include ProjectM Visualization:       no
 - Include Goom Visualization:           no

 =================================================================================
 End Configuration for OpenELEC
 =================================================================================

  BUILD    toolchain (target)
      BUILD    scons (host)
Traceback (most recent call last):
  File "/usr/lib64/python2.7/site.py", line 568, in <module>
    main()
  File "/usr/lib64/python2.7/site.py", line 550, in main
    known_paths = addusersitepackages(known_paths)
  File "/usr/lib64/python2.7/site.py", line 278, in addusersitepackages
    user_site = getusersitepackages()
  File "/usr/lib64/python2.7/site.py", line 253, in getusersitepackages
    user_base = getuserbase() # this will also set USER_BASE
  File "/usr/lib64/python2.7/site.py", line 242, in getuserbase
    from sysconfig import get_config_var
  File "/usr/lib64/python2.7/sysconfig.py", line 10, in <module>
    'stdlib': '{base}/'+sys.lib+'/python{py_version_short}',
AttributeError: 'module' object has no attribute 'lib'
make: *** [system] Error 1

trying a clean make

ghost commented 10 years ago

nope.

build, with mount --bind rather than the symlink

....
Traceback (most recent call last):
  File "/usr/lib64/python2.7/site.py", line 568, in <module>
    main()
  File "/usr/lib64/python2.7/site.py", line 550, in main
    known_paths = addusersitepackages(known_paths)
  File "/usr/lib64/python2.7/site.py", line 278, in addusersitepackages
    user_site = getusersitepackages()
  File "/usr/lib64/python2.7/site.py", line 253, in getusersitepackages
    user_base = getuserbase() # this will also set USER_BASE
  File "/usr/lib64/python2.7/site.py", line 242, in getuserbase
    from sysconfig import get_config_var
  File "/usr/lib64/python2.7/sysconfig.py", line 10, in <module>
    'stdlib': '{base}/'+sys.lib+'/python{py_version_short}',
AttributeError: 'module' object has no attribute 'lib'
make: *** [system] Error 1
ghost commented 10 years ago

I'm suspicious this may be opensuse-specific

here's a possibly related patch

https://build.opensuse.org/package/view_file/openSUSE:13.1/python/python-2.7.5-multilib.patch

not sure yet how it fits into the openelec crosschain

ghost commented 10 years ago

finding/verifying the equivalent python 2.7.3 patch for opensuse arm & cribbing order from here,

https://build.opensuse.org/package/view_file/openSUSE:Factory:ARM/python/python-doc.spec

dropping their patch into openelec

wget -O \
/home/rand/workspace/openelec/packages/lang/Python/patches/Python-2.7.3-009-z-opensuse-python-2.7.5-multilib.patch \
https://build.opensuse.org/source/openSUSE:Factory:ARM/python/python-2.7.5-multilib.patch?rev=88abc8b85bf079f88f44f85a341d6cee

then

make clean
PROJECT=RPi ARCH=arm make

appears to get past the reported fail. build continues ...

ghost commented 10 years ago

the opensuse patch is against configure.ac -- not provided in the openelec python tarball

...
              APPLY PATCH (common)   /home/rand/workspace/openelec/packages/lang/Python/patches/Python-2.7.3-009-z-opensuse-python-2.7.5-multilib.patch
can't find file to patch at input line 5
Perhaps you used the wrong -p or --strip option?
The text leading up to this was:
--------------------------
|Index: Python-2.7.6/configure.ac
|===================================================================
|--- Python-2.7.6.orig/configure.ac     2013-11-10 08:36:41.000000000 +0100
|+++ Python-2.7.6/configure.ac  2014-02-11 20:08:16.265571499 +0100
--------------------------
File to patch:
ghost commented 10 years ago

did a quick test overnite ... all nine cases,

git/{master,4.0,3.2} X working_dir{symlinked to $HOME/xxx, mount --bind to $HOME/xxx, native @ $HOME/xxx}

fail @Python, as reported above

ghost commented 10 years ago

my desktop env includes

env | grep -i python
    PYTHONSTARTUP=/etc/pythonstart
    PYTHONPATH=.:/usr/local/lib64/python2.7:/usr/local/lib/python2.7:/usr/lib64/python2.7:/usr/lib/python2.7

where,

 cat /etc/pythonstart
    import atexit
    import os
    import readline
    import rlcompleter
    historyPath = os.path.expanduser("~/.pyhistory")
    def save_history(historyPath=historyPath):
        import readline
        readline.write_history_file(historyPath)
    if os.path.exists(historyPath):
        readline.set_history_length(10000)
        readline.read_history_file(historyPath)
    atexit.register(save_history)
    readline.parse_and_bind('tab: complete')
    del os, atexit, readline, rlcompleter, save_history, historyPath

testing on OE/master branch,

unset PYTHONPATH PYTHONSTARTUP
make clean

building

PROJECT=RPi ARCH=arm make

sees a LOT of these errors ... Modules/python.o \ -L. -lpython2.7 -lpthread -ldl -lpthread -lutil -lm
/home/rand/workspace/openelec/build.OpenELEC-RPi.arm-devel/toolchain/lib/gcc/armv6zk-openelec-linux-gnueabi/4.8.2/../../../../armv6zk-openelec-linux-gnueabi/bin/ld.gold: warning: skipping incompatible /home/rand/workspace/openelec/build.OpenELEC-RPi.arm-devel/toolchain/lib/libpython2.7.so while searching for python2.7 /home/rand/workspace/openelec/build.OpenELEC-RPi.arm-devel/toolchain/lib/gcc/armv6zk-openelec-linux-gnueabi/4.8.2/../../../../armv6zk-openelec-linux-gnueabi/bin/ld.gold: error: cannot find -lpython2.7 ==> collect2: error: ld returned 1 exit status /home/rand/workspace/openelec/build.OpenELEC-RPi.arm-devel/toolchain/lib/gcc/armv6zk-openelec-linux-gnueabi/4.8.2/../../../../armv6zk-openelec-linux-gnueabi/bin/ld.gold: warning: skipping incompatible /home/rand/workspace/openelec/build.OpenELEC-RPi.arm-devel/toolchain/lib/libpython2.7.so while searching for python2.7 /home/rand/workspace/openelec/build.OpenELEC-RPi.arm-devel/toolchain/lib/gcc/armv6zk-openelec-linux-gnueabi/4.8.2/../../../../armv6zk-openelec-linux-gnueabi/bin/ld.gold: error: cannot find -lpython2.7 ==> collect2: error: ld returned 1 exit status /home/rand/workspace/openelec/build.OpenELEC-RPi.arm-devel/toolchain/lib/gcc/armv6zk-openelec-linux-gnueabi/4.8.2/../../../../armv6zk-openelec-linux-gnueabi/bin/ld.gold: warning: skipping incompatible /home/rand/workspace/openelec/build.OpenELEC-RPi.arm-devel/toolchain/lib/libpython2.7.so while searching for python2.7 /home/rand/workspace/openelec/build.OpenELEC-RPi.arm-devel/toolchain/lib/gcc/armv6zk-openelec-linux-gnueabi/4.8.2/../../../../armv6zk-openelec-linux-gnueabi/bin/ld.gold: error: cannot find -lpython2.7 ==> collect2: error: ld returned 1 exit status ...

reports this

    ...
    Python build finished, but the necessary bits to build these modules were not found:
    _bsddb             _curses            _curses_panel   
    _tkinter           bsddb185           dbm             
    dl                 gdbm               imageop         
    readline           sunaudiodev                        
    To find the necessary bits, look in setup.py in detect_modules() for the module's name.

    Failed to build these modules:
    _bisect            _codecs_cn         _codecs_hk      
    _codecs_iso2022    _codecs_jp         _codecs_kr      
    _codecs_tw         _collections       _csv            
    _ctypes            _ctypes_test       _elementtree    
    _functools         _hashlib           _heapq          
    _hotshot           _io                _json           
    _locale            _lsprof            _multibytecodec 
    _multiprocessing   _random            _socket         
    _sqlite3           _ssl               _struct         
    _testcapi          array              audioop         
    binascii           bz2                cmath           
    cPickle            crypt              cStringIO       
    datetime           fcntl              future_builtins 
    grp                itertools          linuxaudiodev   
    math               mmap               operator        
    ossaudiodev        parser             pyexpat         
    resource           select             spwd            
    strop              syslog             termios         
    time               unicodedata        zlib            
    ...

then continues thru Python-build exit

...
Compiling /home/rand/workspace/openelec/build.OpenELEC-RPi.arm-devel/Python-2.7.3/.install_pkg/usr/lib/python2.7/zipfile.py ...
make[1]: Entering directory `/home/rand/workspace/openelec/build.OpenELEC-RPi.arm-devel/linux-3.13.5'
scripts/kconfig/conf --silentoldconfig Kconfig
...

and, atm, keeps building ...

I'll see how far this gets.

unset'ing the PYTHON* env vars certainly has an effect.

crossbuild shouldn't be inheriting/using parent env's app vars.

either python in crossbuild should be invoked with "-E" to ignore all env vars, or this list, http://docs.python.org/2/using/cmdline.html#environment-variables, should be cleared/locally set/overwritten for the purpose of the build.

stefansaraev commented 10 years ago

good catch. thanks much @rand22 77dc4b8f79276816e3fb1378c91ac854c50942b5 should fix it. let me know if not

ghost commented 10 years ago

verifying, per chat @ irc,

cd $B
git log | head
    commit 3479c2c7f216b58dee7978918d2cba48d7b76f43
    Author: Stefan Saraev <stefan@saraev.ca>
    Date:   Wed Mar 5 19:53:27 2014 +0200

        Mesa: update to Mesa-10.1.0

    commit 5d4c84166810c8729a4df777ac50870c8169f46f
    Author: Stephan Raue <stephan@openelec.tv>
    Date:   Wed Mar 5 17:54:04 2014 +0100

unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS
unset PYTHONSTARTUP
export PYTHONPATH=$B/build.OpenELEC-RPi.arm-devel/toolchain/lib/python2.7
make clean
PROJECT=RPi ARCH=arm make

completes without error

after subsequent,

PROJECT=RPi ARCH=arm make image

end up with

ls -altr /home/rand/workspace/openelec/target/OpenELEC-RPi.arm-devel*
    -rw-r--r--+ 1 root root 6.8M Mar  5 16:05 /home/rand/workspace/openelec/target/OpenELEC-RPi.arm-devel-20140305132135-r17861-g3479c2c.kernel
    -rw-r--r--+ 1 root root  94M Mar  5 16:05 /home/rand/workspace/openelec/target/OpenELEC-RPi.arm-devel-20140305132135-r17861-g3479c2c.system
    -rw-r--r--+ 1 root root 6.8M Mar  5 16:13 /home/rand/workspace/openelec/target/OpenELEC-RPi.arm-devel-20140305161223-r17861-g3479c2c.kernel
    -rw-r--r--+ 1 root root  94M Mar  5 16:13 /home/rand/workspace/openelec/target/OpenELEC-RPi.arm-devel-20140305161223-r17861-g3479c2c.system
    -rw-rw-r--+ 1 root root 104M Mar  5 16:13 /home/rand/workspace/openelec/target/OpenELEC-RPi.arm-devel-20140305161223-r17861-g3479c2c.tar
    -rw-rw-r--+ 1 root root  95M Mar  5 16:13 /home/rand/workspace/openelec/target/OpenELEC-RPi.arm-devel-20140305161223-r17861-g3479c2c.img.gz

as expected

ghost commented 10 years ago

and, with latest pull, including fixes as above,

cd /home/rand/workspace/openelec
git log | head
    commit 77dc4b8f79276816e3fb1378c91ac854c50942b5
    Author: Stefan Saraev <stefan@saraev.ca>
    Date:   Wed Mar 5 23:01:34 2014 +0200

        config/path: unset PYTHONSTARTUP/PYTHONPATH

        tryfix #2964

    commit 3479c2c7f216b58dee7978918d2cba48d7b76f43
    Author: Stefan Saraev <stefan@saraev.ca>

unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS
env | grep -i python
    PYTHONSTARTUP=/etc/pythonstart
    PYTHONPATH=.:/usr/local/lib64/python2.7:/usr/local/lib/python2.7:/usr/lib64/python2.7:/usr/lib/python2.7

PROJECT=RPi ARCH=arm make
PROJECT=RPi ARCH=arm make image

ls -altr /home/rand/workspace/openelec/target/OpenELEC-RPi.arm-devel*
    ...

all GOOD, with no build-time workarounds required

stefansaraev commented 10 years ago

thanks much. closing this then :)