search

niner / inline-python-pm

Inline::Python - Write Perl subs and classes in Python.
https://metacpan.org/release/Inline-Python
20 stars 13 forks source link

Core dump when wrapping numpy 1.9 #9

Closed jeanconn closed 9 years ago

jeanconn commented 9 years ago

Inline::Python dumps core with perl 5.18.2 and numpy 1.9.2. Example that produces the error:

#!/usr/bin/env perl

use Inline PYTHON;
__END__
__PYTHON__
import numpy as np
 *** glibc detected *** perl5.18.2: munmap_chunk(): invalid pointer: 0x00002ba959ec9f74 ***
======= Backtrace: =========
/lib64/libc.so.6(cfree+0x166)[0x3558c758c6]
perl5.18.2(perl_destruct+0x1f8)[0x437338]
perl5.18.2(main+0xc4)[0x41c7b4]
/lib64/libc.so.6(__libc_start_main+0xf4)[0x3558c1d9f4]
perl5.18.2(sin+0xd1)[0x41c639]

The same code works fine with numpy 1.8.2 so I assume this is due to the Numpy C API changes.

niner commented 9 years ago

I cannot reproduce the error with numpy 1.9.2, perl 5.20.1 and Inline::Python 0.48.

Are you using Python 2 or Python 3?

jeanconn commented 9 years ago

Thanks for looking at this. I am using Python 2.7.9 . I'll try a newer Perl. I started with Inline::Python 0.43 and also tested against 0.48, but I'll redo the 0.48 test as well.

jeanconn commented 9 years ago

Tried a source build of Perl 5.20.1. Same core dump. CentOS 5/6. Python 2.7.9. Numpy 1.9.2. Inline 0.76. Inline::Python 0.48. Hmph.

jeanconn commented 9 years ago

Forgot to note, this is all 64 bit.

niner commented 9 years ago

Can you give me the output of perl -V please? Might give me some hint on what the difference in our systems is.

jeanconn commented 9 years ago

Sure. Here's the 5.20.1 I just built and tried:

Summary of my perl5 (revision 5 version 20 subversion 1) configuration:

  Platform:
    osname=linux, osvers=2.6.18-402.el5, archname=x86_64-linux
    uname='linux beep 2.6.18-402.el5 #1 smp tue feb 10 17:49:04 est 2015 x86_64 x86_64 x86_64 gnulinux '
    config_args='-des -Dprefix=/data/aca/arch/x86_64/envs/test -Duserelocatableinc -Dotherlibdirs=.../../lib/perl'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=undef, usemultiplicity=undef
    use64bitint=define, use64bitall=define, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='-O2',
    cppflags='-fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include'
    ccversion='', gccversion='4.1.2 20080704 (Red Hat 4.1.2-55)', gccosandvers=''
    intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
    ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='cc', ldflags ='-fstack-protector -L/usr/local/lib -L/usr/lib64'
    libpth=.../lib /usr/local/lib /usr/lib /lib /lib64 /usr/lib64 /usr/local/lib64
    libs=-lnsl -ldl -lm -lcrypt -lutil -lc
    perllibs=-lnsl -ldl -lm -lcrypt -lutil -lc
    libc=libc-2.5.so, so=so, useshrplib=false, libperl=libperl.a
    gnulibc_version='2.5'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
    cccdlflags='-fPIC', lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector -L/usr/lib64'

Characteristics of this binary (from libperl): 
  Compile-time options: HAS_TIMES PERLIO_LAYERS PERL_DONT_CREATE_GVSV
                        PERL_HASH_FUNC_ONE_AT_A_TIME_HARD PERL_MALLOC_WRAP
                        PERL_NEW_COPY_ON_WRITE PERL_PRESERVE_IVUV
                        USE_64_BIT_ALL USE_64_BIT_INT USE_LARGE_FILES
                        USE_LOCALE USE_LOCALE_COLLATE USE_LOCALE_CTYPE
                        USE_LOCALE_NUMERIC USE_PERLIO USE_PERL_ATOF
  Built under linux
  Compiled at Apr 19 2015 17:43:59
  @INC:
    /data/aca/arch/x86_64/envs/test/lib/perl5/site_perl/5.20.1/x86_64-linux
    /data/aca/arch/x86_64/envs/test/lib/perl5/site_perl/5.20.1
    /data/aca/arch/x86_64/envs/test/lib/perl5/5.20.1/x86_64-linux
    /data/aca/arch/x86_64/envs/test/lib/perl5/5.20.1
    /data/aca/arch/x86_64/envs/test/lib/perl
jeanconn commented 9 years ago

I haven't tracked it down with the debugger. Looks like I can stop the dump if I just delete numpy's multiarray.so so it doesn't import successfully, but that could obviously be a red herring.

jeanconn commented 9 years ago

Ugh. In a totally clean build, this problem has gone away. Sorry for the noise. This was reproducible on two systems, 3 different perl version, and two versions of Inline::Python, but they must have been corrupted in similar fashions.

niner commented 9 years ago

I found out why it sometimes works and sometimes not. It does work when perl is configured with -Accflags='-DPERL_USE_SAFE_PUTENV' like openSUSE's system perl apparently is. Without this flag, Perl will modify the environment variables directly. The glibc error occurs, when perl tries to free the environment variable strings during its cleanup.

So this in fact seems to be the same as https://github.com/niner/inline-python-pm/issues/7 and now I finally found out what's going wrong there!

niner commented 9 years ago

There is actually a way to set the safer behavior at runtime. Implemented it in commit 35db0d3

Please try if this fixes your issues.

jeanconn commented 9 years ago

I was able to break my system again, and it looks like commit 35db0d3 does seem to fix this issue on my system. Thanks!