invalid ELF header on ubuntu

dimitri-yatsenko commented 11 years ago

When I ran mymSetup from the distribution branch on our two ubuntu machines, I got:

MATLAB:invalidMEXFile
Invalid MEX-file '/mnt/lab/users/dimitri/dev/mym-distrib/distribution/mexa64/mym.mexa64':
/mnt/lab/users/dimitri/dev/mym-distrib/distribution/mexa64/libmysqlclient.so.18: invalid ELF header

ahoenselaar commented 11 years ago

That is quite peculiar. Two things:

Can you please try to run everything from a local disk and not from a network drive; there may be issues regarding the noexec mount flag
What version of Ubuntu is this?

dimitri-yatsenko commented 11 years ago

Ubuntu 12.04.2 LTS (GNU/Linux 3.5.0-41-generic x86_64) ran off local disk -- same story

MATLAB:invalidMEXFile
Invalid MEX-file '/home/yatsenko/localdev/mym-distrib/distribution/mexa64/mym.mexa64': /home/yatsenko/localdev/mym-distrib/distribution/mexa64/libmysqlclient.so.18: invalid
ELF header

ahoenselaar commented 11 years ago

Okay, I set up a 64-bit Ubuntu 12.04 LTS and I see the following:

>> cd /ghome/ahoens/mym-bin_v2.6.5-rc2/
>> mymSetup
No connections open
mym is now ready for use.
>> cd distribution/mexa64/
>> ls -l
total 7584
lrwxr-xr-x 1 22770 22770      20 Oct  9 12:07 libmysqlclient.so -> libmysqlclient.so.18
lrwxr-xr-x 1 22770 22770      24 Oct  9 12:07 libmysqlclient.so.18 -> libmysqlclient.so.18.1.0
-rwxr-xr-x 1 22770 22770 7584902 Oct  6 18:03 libmysqlclient.so.18.1.0
lrwxr-xr-x 1 22770 22770      13 Oct  9 12:07 libz.so.1 -> libz.so.1.2.3
-rwxr-xr-x 1 22770 22770   98293 Oct  6 18:03 libz.so.1.2.3
-rwxr-xr-x 1 22770 22770   40285 Oct  6 18:03 mym.mexa64

>> !md5sum *
f951ec4f781a1f80b76c269ebde710ff  libmysqlclient.so
f951ec4f781a1f80b76c269ebde710ff  libmysqlclient.so.18
f951ec4f781a1f80b76c269ebde710ff  libmysqlclient.so.18.1.0
ca3cd0170f67d58c78c809c22a8bc1c1  libz.so.1
ca3cd0170f67d58c78c809c22a8bc1c1  libz.so.1.2.3
36d94c61750808f504b15af976fa4238  mym.mexa64
>> !cat /etc/issue
Ubuntu 12.04.3 LTS \n \l

>> !uname -a
Linux ubuntu 3.8.0-31-generic #46~precise1-Ubuntu SMP Wed Sep 11 18:21:16 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
>> !ldd mym.mexa64
./mym.mexa64: /ghome/ahoens/mym-bin_v2.6.5-rc2/distribution/mexa64/./libz.so.1: no version information available (required by ./mym.mexa64)
    linux-vdso.so.1 =>  (0x00007fff785d8000)
    libmysqlclient.so.18 => /ghome/ahoens/mym-bin_v2.6.5-rc2/distribution/mexa64/./libmysqlclient.so.18 (0x00007f824f648000)
    libz.so.1 => /ghome/ahoens/mym-bin_v2.6.5-rc2/distribution/mexa64/./libz.so.1 (0x00007f824f531000)
    libmx.so => /sw/matlab/R2012a/bin/glnxa64/libmx.so (0x00007f824f277000)
    libmex.so => /sw/matlab/R2012a/bin/glnxa64/libmex.so (0x00007f824f055000)
    libmat.so => /sw/matlab/R2012a/bin/glnxa64/libmat.so (0x00007f824ee06000)
    libstdc++.so.6 => /sw/matlab/R2012a/sys/os/glnxa64/libstdc++.so.6 (0x00007f824eaf7000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f824e7eb000)
    libgcc_s.so.1 => /sw/matlab/R2012a/sys/os/glnxa64/libgcc_s.so.1 (0x00007f824e5d4000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f824e3b7000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f824dff8000)
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f824ddf3000)
    librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f824dbeb000)
    libmwresource_core.so => /sw/matlab/R2012a/bin/glnxa64/libmwresource_core.so (0x00007f824d9e8000)
    libmwi18n.so => /sw/matlab/R2012a/bin/glnxa64/libmwi18n.so (0x00007f824d754000)
    libut.so => /sw/matlab/R2012a/bin/glnxa64/libut.so (0x00007f824d460000)
    libmwfl.so => /sw/matlab/R2012a/bin/glnxa64/libmwfl.so (0x00007f824d168000)
    libmwMATLAB_res.so => /sw/matlab/R2012a/bin/glnxa64/libmwMATLAB_res.so (0x00007f824cc6c000)
    libboost_date_time.so.1.44.0 => /sw/matlab/R2012a/bin/glnxa64/libboost_date_time.so.1.44.0 (0x00007f824ca5a000)
    libboost_system.so.1.44.0 => /sw/matlab/R2012a/bin/glnxa64/libboost_system.so.1.44.0 (0x00007f824c856000)
    libboost_thread.so.1.44.0 => /sw/matlab/R2012a/bin/glnxa64/libboost_thread.so.1.44.0 (0x00007f824c63f000)
    libicudata.so.44 => /sw/matlab/R2012a/bin/glnxa64/libicudata.so.44 (0x00007f824b5ff000)
    libicuuc.so.44 => /sw/matlab/R2012a/bin/glnxa64/libicuuc.so.44 (0x00007f824b29a000)
    libicui18n.so.44 => /sw/matlab/R2012a/bin/glnxa64/libicui18n.so.44 (0x00007f824ae9d000)
    libicuio.so.44 => /sw/matlab/R2012a/bin/glnxa64/libicuio.so.44 (0x00007f824ac90000)
    libtbb.so.2 => /sw/matlab/R2012a/bin/glnxa64/libtbb.so.2 (0x00007f824ab50000)
    libtbbmalloc.so.2 => /sw/matlab/R2012a/bin/glnxa64/libtbbmalloc.so.2 (0x00007f824aa27000)
    libmwservices.so => /sw/matlab/R2012a/bin/glnxa64/libmwservices.so (0x00007f824a594000)
    libmwmpath.so => /sw/matlab/R2012a/bin/glnxa64/libmwmpath.so (0x00007f824a351000)
    libmwm_dispatcher.so => /sw/matlab/R2012a/bin/glnxa64/libmwm_dispatcher.so (0x00007f824a0b6000)
    libhdf5_hl.so.6 => /sw/matlab/R2012a/bin/glnxa64/libhdf5_hl.so.6 (0x00007f8249e89000)
    libhdf5.so.6 => /sw/matlab/R2012a/bin/glnxa64/libhdf5.so.6 (0x00007f82499f8000)
    /lib64/ld-linux-x86-64.so.2 (0x00007f824fe19000)
    libboost_filesystem.so.1.44.0 => /sw/matlab/R2012a/bin/glnxa64/libboost_filesystem.so.1.44.0 (0x00007f82497d5000)
    libexpat.so.1 => /sw/matlab/R2012a/bin/glnxa64/libexpat.so.1 (0x00007f82495ad000)
    libboost_regex.so.1.44.0 => /sw/matlab/R2012a/bin/glnxa64/libboost_regex.so.1.44.0 (0x00007f824929f000)
    libboost_signals.so.1.44.0 => /sw/matlab/R2012a/bin/glnxa64/libboost_signals.so.1.44.0 (0x00007f824908c000)
    libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1 (0x00007f8248e53000)
    libmwmfl_scalar.so => /sw/matlab/R2012a/bin/glnxa64/libmwmfl_scalar.so (0x00007f8248c41000)
    libmwmlutil.so => /sw/matlab/R2012a/bin/glnxa64/libmwmlutil.so (0x00007f8248582000)
    libmwMATLAB_settings_res.so => /sw/matlab/R2012a/bin/glnxa64/libmwMATLAB_settings_res.so (0x00007f824836f000)
    libboost_serialization.so.1.44.0 => /sw/matlab/R2012a/bin/glnxa64/libboost_serialization.so.1.44.0 (0x00007f82480fc000)
    libncurses.so.5 => /lib/x86_64-linux-gnu/libncurses.so.5 (0x00007f8247eda000)
    libmwxmlcore.so => /sw/matlab/R2012a/bin/glnxa64/libmwxmlcore.so (0x00007f8247c60000)
    libxerces-c.so.27 => /sw/matlab/R2012a/bin/glnxa64/libxerces-c.so.27 (0x00007f8247663000)
    libtinfo.so.5 => /lib/x86_64-linux-gnu/libtinfo.so.5 (0x00007f824743b000)
>>

The peculiar aspect is that the error is related to libmysql and not the MEX file itself. I'm distributing the official binary connector from MySQL.

I did encounter issues with this build on RHEL 6 due to the glibc version. So I just pushed an updated binary that was built on an older Linux release; maybe that will help.

dimitri-yatsenko commented 11 years ago

thanks for the checksums. For some reason the .so files had been modified by something. The git status said something like 'type changed' for these files. I am not sure what that meant.

After I did git checkout -- on these files, mym worked. But I still don't know what changed these files. I didn't.

ahoenselaar commented 11 years ago

The evil bit fairy at work, haxoring your binaries...

dimitri-yatsenko commented 11 years ago

yes, there is something with the library located on the network drive vs. locally.

The local copy works normally but the network copy gives me this

MATLAB:invalidMEXFile
Invalid MEX-file '/mnt/lab/users/dimitri/dev/mym/distribution/mexa64/mym.mexa64':
/mnt/lab/users/dimitri/dev/mym/distribution/mexa64/libmysqlclient.so.18: file too short

yatsenko@at-compute002:~/dev/mym/distribution/mexa64$ ls -l
total 7546
-rwxrwxr-x 0 root users      20 Oct  9 20:50 libmysqlclient.so
-rwxrwxr-x 0 root users      24 Oct  9 20:50 libmysqlclient.so.18
-rwxrwxr-x 0 root users 7584902 Oct  9 20:50 libmysqlclient.so.18.1.0
-rwxrwxr-x 0 root users      13 Oct  9 20:50 libz.so.1
-rwxrwxr-x 0 root users   98293 Oct  9 20:50 libz.so.1.2.3
-rwxrwxr-x 0 root users   40308 Oct  9 20:50 mym.mexa64

yatsenko@at-compute002:~/dev/mym/distribution/mexa64$ git status
# On branch distribution
nothing to commit (working directory clean)

yatsenko@at-compute002:~/dev/mym/distribution/mexa64$ md5sum *
86deb0d8c7dc21b18ae3a6285e9df6fb  libmysqlclient.so
c4d891fdc81c35ffae0073c8a51ee723  libmysqlclient.so.18
f951ec4f781a1f80b76c269ebde710ff  libmysqlclient.so.18.1.0
106d02b64f844b6c3335b7c7f08f61a9  libz.so.1
ca3cd0170f67d58c78c809c22a8bc1c1  libz.so.1.2.3
dc443a6bb452182fa1831764ce7bf943  mym.mexa64

yatsenko@at-compute002:~/dev/mym/distribution/mexa64$ ldd mym.mexa64 
./mym.mexa64: error while loading shared libraries: /mnt/lab/users/dimitri/dev/mym/distribution/mexa64/./libmysqlclient.so.18: file too short

I am confused and running from a local copy for now.

dimitri-yatsenko commented 11 years ago

Thanks by the way. The new file organization and the distribution branch make things much clearer.

ahoenselaar commented 11 years ago

Sure! From your ls -l above, it is clear that the symlinks did not get created correctly. My wild guess is that /mnt/lab is a Windows shared folder and thus does not support symlinks.

In this particular scenario, the easiest solution would be to replace the symlinked files with the full binary, such that

libmysqlclient.so, libmysqlclient.so.18 and libmysqlclient.so.18.1.0 all are normal files with identical content and 7.5 MB size.

ahoenselaar commented 11 years ago

The only file you really need is probably libmysqlclient.so.18

datajoint / mym

invalid ELF header on ubuntu #5