search

RhysU / ESIO

The ExaScale IO (ESIO) library provides simple, high throughput input and output of structured data sets using parallel HDF5. ESIO is designed to support reading and writing turbulence simulation restart files within C, C++, and modern Fortran applications.
https://rhysu.github.io/ESIO/
GNU Lesser General Public License v2.1
11 stars 2 forks source link

Compiler dependency #5

Open M-K-Lee opened 7 years ago

M-K-Lee commented 7 years ago

@dmcdougall

I tried to install ESIO-0.1.on Wrangler, TACC.

First I configured with "CC=mpicc FC=mpif90 CXX=mpicxx". It configured okay, but showed error when compiling the code with "MAKE". Next, I configured with "CC=icc FC=ifort CXX=icpc", and it worked just fine.

More specific information is as follows.

~$ module list

Currently Loaded Modules: 1) TACC-paths 2) Linux 3) cluster-paths 4) intel/15.0.3 5) mvapich2/2.1 6) cluster 7) TACC 8) phdf5/1.8.14

~$ ../configure --prefix=(some_local_dir) --with-hdf5=$TACC_HDF5_BIN/h5pcc CFLAGS="-fPIC" CPPFLAGS="-fPIC" CXXFLAGS="-fPIC" FCFLAGS="-fPIC" LDFLAGS="-L/usr/lib64" CC=mpicc FC=mpif90 CXX=mpicxx ..................long details............. ----------------------------------- SUMMARY -----------------------------------

Package version............... : esio-0.1.9

C++ compiler.................. : mpicxx C++ compiler flags............ : -fPIC -g -Wno-unknown-pragmas C compiler.................... : mpicc C compiler flags.............. : -fPIC -g -traceback -Wall -std=gnu99 -Wcheck -Wpointer-arith -wd424 -wd981 -Wno-unknown-pragmas Fortran compiler ............. : mpif90 Fortran compiler flags........ : -fPIC -g -warn all -traceback Install dir................... : /home/01545/pro0104/SOFTWARE/ESIO-0.1.9 HDF CPP flags................. : HDF C flags................. : HDF LDFLAGS flags............. : Build user.................... : pro0104 Build host.................... : login1.wrangler.tacc.utexas.edu Configure date................ : 2017-08-15 14:40 Build architecture............ : x86_64-unknown-linux-gnu SVN revision number........... :

~$ make ..............long compilation lines.............................. libtool: link: mpif90 -I.. -I../.. -I../esio -fPIC -g -warn all -traceback -Wl,--as-needed -o .libs/concepts2 concepts2.o -Wl,-rpath -Wl,/opt/apps/intel15/mvapich2_2_1/phdf5/1.8.14/x86_64/lib -L/usr/lib64 ../esio/.libs/libesiof.so -L/opt/apps/intel15/mvapich2_2_1/phdf5/1.8.14/x86_64/lib /opt/apps/intel15/mvapich2/2.1/lib/libmpifort.so -L/usr/lib64/ -L/lib /opt/apps/intel15/mvapich2/2.1/lib/libmpi.so -lxml2 -libmad -libumad -libverbs -lrt /opt/apps/intel15/mvapich2_2_1/phdf5/1.8.14/x86_64/lib/libhdf5_hl.so /opt/apps/intel15/mvapich2_2_1/phdf5/1.8.14/x86_64/lib/libhdf5.so /opt/apps/intel15/mvapich2_2_1/phdf5/1.8.14/x86_64/lib/libsz.so -lz -ldl -lm -Wl,-rpath -Wl,/home/01545/pro0104/SOFTWARE/ESIO-0.1.9/lib -Wl,-rpath -Wl,/opt/apps/intel15/mvapich2/2.1/lib -Wl,-rpath -Wl,/opt/apps/intel15/mvapich2_2_1/phdf5/1.8.14/x86_64/lib ld: cannot find -libverbs make[3]: [concepts2] Error 1 make[3]: Leaving directory `/home/01545/pro0104/SOFTWARE/esio-0.1.9/build/examples' make[2]: [all] Error 2 make[2]: Leaving directory /home/01545/pro0104/SOFTWARE/esio-0.1.9/build/examples' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory/home/01545/pro0104/SOFTWARE/esio-0.1.9/build' make: *** [all] Error 2

RhysU commented 7 years ago

What happens if you feed it h5pcc and the like for CC, CXX, etc.?

RhysU commented 7 years ago

Any news on attempting to force h5pcc as mentioned above?

M-K-Lee commented 7 years ago

Rhys,

I am in the conference. I will test it when I get back to Austin.

MK

On Tue, Aug 22, 2017 at 2:33 AM, Rhys Ulerich notifications@github.com wrote:

Any news on attempting to force h5pcc as mentioned above?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/RhysU/ESIO/issues/5#issuecomment-323886550, or mute the thread https://github.com/notifications/unsubscribe-auth/ANoMpupLr0Vd3IN_QKqWIFVkBOIS4YgBks5saiHtgaJpZM4O4BTx .

-- Best regards, MyoungKyu(M.K.) Lee

M-K-Lee commented 7 years ago

Rhys,

FYI, Damon generously agreed to work on the issue. (Thank you damon.)

Best, MK

On Tue, Aug 22, 2017 at 7:46 AM, Myoungkyu Lee leemyoungkyu@gmail.com wrote:

Rhys,

I am in the conference. I will test it when I get back to Austin.

MK

On Tue, Aug 22, 2017 at 2:33 AM, Rhys Ulerich notifications@github.com wrote:

Any news on attempting to force h5pcc as mentioned above?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/RhysU/ESIO/issues/5#issuecomment-323886550, or mute the thread https://github.com/notifications/unsubscribe-auth/ANoMpupLr0Vd3IN_QKqWIFVkBOIS4YgBks5saiHtgaJpZM4O4BTx .

-- Best regards, MyoungKyu(M.K.) Lee

-- Best regards, MyoungKyu(M.K.) Lee

nicholasmalaya commented 7 years ago

So this ticket can be closed, right?

dmcdougall commented 7 years ago 
$ ../configure --prefix=$HOME/esio CFLAGS="-fPIC" CPPFLAGS="-fPIC" CXXFLAGS="-fPIC" FCFLAGS="-fPIC" CC=h5pcc FC=h5pfc CXX=mpicxx && make

... make coffee ...

libtool: link: mpicc -O3 -fPIC -fPIC -g -traceback -Wall -std=gnu99 -Wcheck -Wpointer-arith -wd424 -wd981 -Wno-unknown-pragmas -Wl,--as-needed -o .libs/concepts1 concepts1.o -Wl,-rpath -Wl,/opt/apps/intel15/mvapich2_2_1/phdf5/1.8.14/x86_64/lib  ../esio/.libs/libesio.so -L/opt/apps/intel15/mvapich2_2_1/phdf5/1.8.14/x86_64/lib /opt/apps/intel15/mvapich2_2_1/phdf5/1.8.14/x86_64/lib/libhdf5_hl.so /opt/apps/intel15/mvapich2_2_1/phdf5/1.8.14/x86_64/lib/libhdf5.so /opt/apps/intel15/mvapich2_2_1/phdf5/1.8.14/x86_64/lib/libsz.so -lz -ldl -lm -Wl,-rpath -Wl,/home/02664/damon/esio/lib -Wl,-rpath -Wl,/opt/apps/intel15/mvapich2_2_1/phdf5/1.8.14/x86_64/lib
concepts1.o: In function `main':
/home/02664/damon/src/ESIO-release-0.1.9/build/examples/../../examples/concepts1.c:13: undefined reference to `esio_handle_initialize'
/home/02664/damon/src/ESIO-release-0.1.9/build/examples/../../examples/concepts1.c:14: undefined reference to `esio_file_create'
/home/02664/damon/src/ESIO-release-0.1.9/build/examples/../../examples/concepts1.c:16: undefined reference to `esio_string_set'
/home/02664/damon/src/ESIO-release-0.1.9/build/examples/../../examples/concepts1.c:19: undefined reference to `esio_attribute_write_int'
/home/02664/damon/src/ESIO-release-0.1.9/build/examples/../../examples/concepts1.c:22: undefined reference to `esio_line_establish'
/home/02664/damon/src/ESIO-release-0.1.9/build/examples/../../examples/concepts1.c:23: undefined reference to `esio_line_write_double'
/home/02664/damon/src/ESIO-release-0.1.9/build/examples/../../examples/concepts1.c:25: undefined reference to `esio_file_close'
/home/02664/damon/src/ESIO-release-0.1.9/build/examples/../../examples/concepts1.c:26: undefined reference to `esio_handle_finalize'

Couldn't find a C++ equivalent for h5pcc or h5pfc.

dmcdougall commented 7 years ago

Also getting inconsistencies in whether or not -fPIC is used.

RhysU commented 7 years ago

Have a machine where I can try it out personally? IIRC shared hdf5 Fortran was a trick.

On Sep 4, 2017 2:52 PM, "Damon McDougall" notifications@github.com wrote:

Also getting inconsistencies in whether or not -fPIC is used.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/RhysU/ESIO/issues/5#issuecomment-327014841, or mute the thread https://github.com/notifications/unsubscribe-auth/AAFNq21tkGkMqjz3pVgNCjxhxKWoIoNLks5sfEbmgaJpZM4O4BTx .

dmcdougall commented 7 years ago

Not sure. I'll see if I can get a recreate on an ICES machine; do you still have access to any of those?

RhysU commented 7 years ago

I doubt it. Let me see what I can recreate on my machine and then we can compare notes.

On Sep 4, 2017 3:36 PM, "Damon McDougall" notifications@github.com wrote:

Not sure. I'll see if I can get a recreate on an ICES machine; do you still have access to any of those?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/RhysU/ESIO/issues/5#issuecomment-327019263, or mute the thread https://github.com/notifications/unsubscribe-auth/AAFNq2TyaNc36rtPPpofbgulRxt2Ds3tks5sfFEfgaJpZM4O4BTx .

dmcdougall commented 7 years ago

Alright. Let me know what I can try from my end; I realise you most certainly don't have access to Intel's compilers and whatnot.

RhysU commented 7 years ago

How does something like the following treat you on branch develop?

$ cd $HOME/Build
$ wget https://support.hdfgroup.org/ftp/HDF5/current18/src/hdf5-1.8.19.tar.bz2
$ tar xjvf hdf5-1.8.19.tar.bz2
$ cd hdf5-1.8.19
$ ./configure --prefix=$HOME/Stow/hdf5-1.8.19 --enable-parallel --enable-shared
$ make -j
$ make install
$ cd ..
$ export PATH=$HOME/Stow/hdf5-1.8.19/bin:$PATH
$ git clone https://github.com/RhysU/ESIO.git esio
$ cd esio
$ git checkout develop
$ ./bootstrap
$ cd ../
$ mkdir esio-gcc
$ cd esio-gcc
$ ../esio/configure --prefix=$HOME/Stow/esio-0.1.9
$ make -j
$ make check

I unhacked some stupid youthful hackery in b5d3e1ca9ced. Notice neither Fortran nor C++ HDF5 wrappers required.

FWIW, on Ubuntu 17.04 couldn't use default HDF5 per #4. I also see a stack smashing complaint come back on one Fortran test case, which is new to me, but is wayyyyyy down in OpenMPI.

dmcdougall commented 7 years ago

That ninja comment edit confused me. I'll let you know shortly.

dmcdougall commented 7 years ago

Compiled successfully. I also tried setting CC=mpicc and the like and that also compiled successfully. Running make check now.

dmcdougall commented 7 years ago

I'm seeing a nontrivial number of test failures. Here's one of them:

login1.wrangler(16)$ cat tests/basic.sh.log
mpiexec -np 1 ./basic
success_code ...................................................... PASS
file_create_and_open ..............................................HDF5-DIAG: Error detected in HDF5 (1.8.14) MPI-process 0:
  #000: H5F.c line 522 in H5Fcreate(): unable to create file
    major: File accessibilty
    minor: Unable to open file
  #001: H5Fint.c line 990 in H5F_open(): unable to open file: time = Wed Sep  6 13:55:49 2017
, name = './test.basic.c.1A0tsj', tent_flags = 13
    major: File accessibilty
    minor: Unable to open file
  #002: H5FD.c line 992 in H5FD_open(): open failed
    major: Virtual File Layer
    minor: Unable to initialize object
  #003: H5FDmpio.c line 1061 in H5FD_mpio_open(): MPI_File_open failed
    major: Internal error (too specific to document in detail)
    minor: Some MPI function failed
  #004: H5FDmpio.c line 1061 in H5FD_mpio_open(): File does not exist, error stack:
ADIO_RESOLVEFILETYPE_FNCALL(473): Invalid file name ./test.basic.c.1A0tsj
    major: Internal error (too specific to document in detail)
    minor: MPI Error String
esio: ../../esio/esio.c:506: ERROR: Unable to create file
Default esio error handler invoked.
[cli_0]: aborting job:
application called MPI_Abort(MPI_COMM_WORLD, 1) - process 0

===================================================================================
=   BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
=   PID 80664 RUNNING AT login1.wrangler.tacc.utexas.edu
=   EXIT CODE: 1
=   CLEANING UP REMAINING PROCESSES
=   YOU CAN IGNORE THE BELOW CLEANUP MESSAGES
===================================================================================
dmcdougall commented 7 years ago

find .. -iname "test.basic.c.1A0tsj" doesn't return anything, so this file is missing.

Edit: I'm in a build directory, one level below the repo root.

RhysU commented 7 years ago

Unable to create file looks fishy-- can you manually run a test or two? They're driven from bash scripts underneath the tests/ directory.

dmcdougall commented 7 years ago

Same thing:

c252-135.wrangler(83)$ pwd
/home/02664/damon/git/ESIO/build/tests
c252-135.wrangler(84)$ ibrun ./basic  # For this case, this is basically mpiexec -np 1
TACC: Starting up job 52033
TACC: Setting up parallel environment for MVAPICH2+mpispawn.
TACC: Starting parallel tasks...
success_code ...................................................... PASS
file_create_and_open ..............................................HDF5-DIAG: Error detected in HDF5 (1.8.14) MPI-process 0:
  #000: H5F.c line 522 in H5Fcreate(): unable to create file
    major: File accessibilty
    minor: Unable to open file
  #001: H5Fint.c line 990 in H5F_open(): unable to open file: time = Mon Sep 11 17:13:59 2017
, name = '/tmp/test.basic.c.SWcAXO', tent_flags = 13
    major: File accessibilty
    minor: Unable to open file
  #002: H5FD.c line 992 in H5FD_open(): open failed
    major: Virtual File Layer
    minor: Unable to initialize object
  #003: H5FDmpio.c line 1061 in H5FD_mpio_open(): MPI_File_open failed
    major: Internal error (too specific to document in detail)
    minor: Some MPI function failed
  #004: H5FDmpio.c line 1061 in H5FD_mpio_open(): File does not exist, error stack:
ADIO_RESOLVEFILETYPE_FNCALL(473): Invalid file name /tmp/test.basic.c.SWcAXO
    major: Internal error (too specific to document in detail)
    minor: MPI Error String
esio: ../../esio/esio.c:507: ERROR: Unable to create file
Default esio error handler invoked.
[cli_0]: aborting job:
application called MPI_Abort(MPI_COMM_WORLD, 1) - process 0
[c252-135.wrangler.tacc.utexas.edu:mpispawn_0][child_handler] MPI process (rank: 0, pid: 24417) exited with status 1
TACC: MPI job exited with code: 1

TACC: Shutdown complete. Exiting.

Making a directory called foo and setting ESIO_TEST_OUTPUT_DIR=foo yields the same error message.

I also can't gdb the test binary:

c252-135.wrangler(85)$ ../libtool --mode=execute gdb --tui ./basic

(gdb) start
Temporary breakpoint 1 at 0x406e55: file ../../tests/basic.c, line 64.
Starting program: /home/02664/damon/git/ESIO/build/tests/.libs/basic
Cannot create process: No such process
During startup program exited with code 127.

Line 64 of basic.c is the FCT_BGN() macro, and I have no idea what that is doing.

To preserve my sanity, I'm writing a small C program that creates an HDF5 file "in parallel". The quotes mean that, simply, this is a one-process MPI code.

dmcdougall commented 7 years ago

To preserve my sanity, I'm writing a small C program that creates an HDF5 file "in parallel". The quotes mean that, simply, this is a one-process MPI code.

My hello world H5Fcreate in parallel doesn't work, so this is a problem on Wrangler, not with ESIO.

RhysU commented 7 years ago

Just /tmp not writeable either from the shell or inside an mpiexec? Does tweaking TMPDIR to, say, $HOME/tmp change anything?

dmcdougall commented 7 years ago

It's filesystem-independent; I'm pretty sure the phdf5 install on Wrangler is just borked.