search

rumpkernel / rumprun-packages

Ready-made packages of software for running on the Rumprun unikernel
Other
202 stars 80 forks source link

openfoam compilation problem #118

Open justinc1 opened 7 years ago

justinc1 commented 7 years ago

I'm trying to run OpenFOAM on rumpkernel. At the moment, compilation fails, and I have problem understanding why, and more precisely, how should I modify build procedures to make them rump-compatilble. I guess issues is a wrong place for such novice question, but -I don't know where else could I ask for help.

Basic OpenFOAM (a bit outdated 2.4.0 version) compilation/installation is explained on https://openfoamwiki.net/index.php/Installation/Linux/OpenFOAM-2.4.0/Fedora. They use their wmake utility, since plain make and friends are supposed to be to weak for their requirements. Compilation results on host are a bunch of .so libraries. This works, so I do have prerequisite devel libraries etc on my system.

When trying to compile for rumpkernel, the first thing which fails is utility libPstream.so. I added some 'set -x' to see, which gcc command fails, and then tried to compile libPstream.so only:

cd src/Pstream/dummy/Make/linux64GccDPOpt
wmake  # -> error
#
make -f /home/justin_cinkelj/devel/mikelangelo/rumprun-packages/openfoam/build/OpenFOAM-2.4.0/wmake/Makefile MAKE_DIR=Make INCLUDE_DEPS=Make/linux64GccDPOpt/includeDeps libso
/home/justin_cinkelj/devel/mikelangelo/rumprun/./rumprun/libexec/rumprun-x86_64-hw/x86_64--netbsd-gcc -mno-red-zone -no-integrated-cpp --sysroot /home/justin_cinkelj/devel/mikelangelo/rumprun/./rumprun/rumprun-x86_64 -specs=/home/justin_cinkelj/devel/mikelangelo/rumprun/./rumprun/rumprun-x86_64/lib/specs-compile_or_ferment -Wl,-r -Wl,-u,main -Dlinux64 -DWM_DP -Wall -Wextra -Wno-unused-parameter -Wold-style-cast -Wnon-virtual-dtor -O3 -DNoRepository -ftemplate-depth-100 -IlnInclude -I. -I/home/justin_cinkelj/devel/mikelangelo/rumprun-packages/openfoam/build/OpenFOAM-2.4.0/src/OpenFOAM/lnInclude -I/home/justin_cinkelj/devel/mikelangelo/rumprun-packages/openfoam/build/OpenFOAM-2.4.0/src/OSspecific/POSIX/lnInclude -fPIC -shared -Xlinker --add-needed -Xlinker --no-as-needed UPstream.o UIPread.o UOPwrite.o -L/home/justin_cinkelj/devel/mikelangelo/rumprun-packages/openfoam/build/OpenFOAM-2.4.0/platforms/linux64GccDPOpt/lib -o /home/justin_cinkelj/devel/mikelangelo/rumprun-packages/openfoam/build/OpenFOAM-2.4.0/platforms/linux64GccDPOpt/lib/dummy/libPstream.so /home/justin_cinkelj/devel/mikelangelo/rumprun/./rumprun/rumprun-x86_64/share/x86_64-rumprun-netbsd-recipe.s
# succeded, I got ELF 64-bit LSB relocatable for 'file libPstream.so'

/home/justin_cinkelj/devel/mikelangelo/rumprun/./rumprun/libexec/rumprun-x86_64-hw/brprintmetainfo /home/justin_cinkelj/devel/mikelangelo/rumprun-packages/openfoam/build/OpenFOAM-2.4.0/platforms/linux64GccDPOpt/lib/dummy/libPstream.so

/home/justin_cinkelj/devel/mikelangelo/rumprun/./rumprun/libexec/rumprun-x86_64-hw/x86_64--netbsd-gcc -mno-red-zone -no-integrated-cpp --sysroot /home/justin_cinkelj/devel/mikelangelo/rumprun/./rumprun/rumprun-x86_64 -specs=/home/justin_cinkelj/devel/mikelangelo/rumprun/./rumprun/rumprun-x86_64/lib/specs-stub /home/justin_cinkelj/devel/mikelangelo/rumprun-packages/openfoam/build/OpenFOAM-2.4.0/platforms/linux64GccDPOpt/lib/dummy/libPstream.so -o /dev/null
# many errors
/home/justin_cinkelj/devel/mikelangelo/rumprun-packages/openfoam/build/OpenFOAM-2.4.0/platforms/linux64GccDPOpt/lib/dummy/libPstream.so: In function `Foam::UPstream::init(int&, char**&)':
(.text+0x14): undefined reference to `Foam::FatalError'
/home/justin_cinkelj/devel/mikelangelo/rumprun-packages/openfoam/build/OpenFOAM-2.4.0/platforms/linux64GccDPOpt/lib/dummy/libPstream.so: In function `Foam::UPstream::init(int&, char**&)':
(.text+0x2f): undefined reference to `Foam::error::operator()(char const*, char const*, int)'
/home/justin_cinkelj/devel/mikelangelo/rumprun-packages/openfoam/build/OpenFOAM-2.4.0/platforms/linux64GccDPOpt/lib/dummy/libPstream.so: In function `Foam::UPstream::init(int&, char**&)':
...

Is the last command trying to assemble the binary for rumprun-bake step, and fails as required libs are missing? In tutorial (https://github.com/rumpkernel/wiki/wiki/Tutorial:-Building-Rumprun-Unikernels) I saw only compilation of single-file app. Now, with complex multi-libs app, I don't know what is build system supposed to do.

anttikantee commented 7 years ago

On 19/09/16 11:05, justinc1 wrote:

I'm trying to run OpenFOAM on rumpkernel. At the moment, compilation fails, and I have problem understanding why, and more precisely, how should I modify build procedures to make them rump-compatilble. I guess issues is a wrong place for such novice question, but -I don't know where else could I ask for help.

It's fine to have such discussions here (or on the mailing list).

Basic OpenFOAM (a bit outdated 2.4.0 version) compilation/installation is explained on https://openfoamwiki.net/index.php/Installation/Linux/OpenFOAM-2.4.0/Fedora. They use their wmake utility, since plain make and friends are supposed to be to weak for their requirements. Compilation results on host are a bunch of .so libraries. This works, so I do have prerequisite devel libraries etc on my system.

When trying to compile for rumpkernel, the first thing which fails is utility libPstream.so. I added some 'set -x' to see, which gcc command fails, and then tried to compile libPstream.so only:

Shared libraries are not supported by Rumprun. Can you try to convince OpenFOAM to build/use static libraries?

cd src/Pstream/dummy/Make/linux64GccDPOpt
wmake  # -> error
#
make -f /home/justin_cinkelj/devel/mikelangelo/rumprun-packages/openfoam/build/OpenFOAM-2.4.0/wmake/Makefile MAKE_DIR=Make INCLUDE_DEPS=Make/linux64GccDPOpt/includeDeps libso
/home/justin_cinkelj/devel/mikelangelo/rumprun/./rumprun/libexec/rumprun-x86_64-hw/x86_64--netbsd-gcc -mno-red-zone -no-integrated-cpp --sysroot /home/justin_cinkelj/devel/mikelangelo/rumprun/./rumprun/rumprun-x86_64 -specs=/home/justin_cinkelj/devel/mikelangelo/rumprun/./rumprun/rumprun-x86_64/lib/specs-compile_or_ferment -Wl,-r -Wl,-u,main -Dlinux64 -DWM_DP -Wall -Wextra -Wno-unused-parameter -Wold-style-cast -Wnon-virtual-dtor -O3 -DNoRepository -ftemplate-depth-100 -IlnInclude -I. -I/home/justin_cinkelj/devel/mikelangelo/rumprun-packages/openfoam/build/OpenFOAM-2.4.0/src/OpenFOAM/lnInclude -I/home/justin_cinkelj/devel/mikelangelo/rumprun-packages/openfoam/build/OpenFOAM-2.4.0/src/OSspecific/POSIX/lnInclude -fPIC -shared -Xlinker --add-needed -Xlinker --no-as-needed UPstream.o UIPread.o UOPwrite.o -L/home/justin_cinkelj/devel/mikelangelo/rumprun-packages/openfoam/build/OpenFOAM-2.4.0/platforms/linux64GccDPOpt/lib -o /home/justin_cinkelj/devel/mikelangelo/rumprun-packages/openfoam/build/OpenFOAM-2.4.0/platforms/linux64GccDPOpt/lib/dummy/libPstream.so /home/justin_cinkelj/devel/mikelangelo/rumprun/./rumprun/rumprun-x86_64/share/x86_64-rumprun-netbsd-recipe.s
# succeded, I got ELF 64-bit LSB relocatable for 'file libPstream.so'

/home/justin_cinkelj/devel/mikelangelo/rumprun/./rumprun/libexec/rumprun-x86_64-hw/brprintmetainfo /home/justin_cinkelj/devel/mikelangelo/rumprun-packages/openfoam/build/OpenFOAM-2.4.0/platforms/linux64GccDPOpt/lib/dummy/libPstream.so

/home/justin_cinkelj/devel/mikelangelo/rumprun/./rumprun/libexec/rumprun-x86_64-hw/x86_64--netbsd-gcc -mno-red-zone -no-integrated-cpp --sysroot /home/justin_cinkelj/devel/mikelangelo/rumprun/./rumprun/rumprun-x86_64 -specs=/home/justin_cinkelj/devel/mikelangelo/rumprun/./rumprun/rumprun-x86_64/lib/specs-stub /home/justin_cinkelj/devel/mikelangelo/rumprun-packages/openfoam/build/OpenFOAM-2.4.0/platforms/linux64GccDPOpt/lib/dummy/libPstream.so -o /dev/null
# many errors
/home/justin_cinkelj/devel/mikelangelo/rumprun-packages/openfoam/build/OpenFOAM-2.4.0/platforms/linux64GccDPOpt/lib/dummy/libPstream.so: In function `Foam::UPstream::init(int&, char**&)':
(.text+0x14): undefined reference to `Foam::FatalError'
/home/justin_cinkelj/devel/mikelangelo/rumprun-packages/openfoam/build/OpenFOAM-2.4.0/platforms/linux64GccDPOpt/lib/dummy/libPstream.so: In function `Foam::UPstream::init(int&, char**&)':
(.text+0x2f): undefined reference to `Foam::error::operator()(char const*, char const*, int)'
/home/justin_cinkelj/devel/mikelangelo/rumprun-packages/openfoam/build/OpenFOAM-2.4.0/platforms/linux64GccDPOpt/lib/dummy/libPstream.so: In function `Foam::UPstream::init(int&, char**&)':
...

Is the last command trying to assemble the binary for rumprun-bake step, and fails as required libs are missing? In tutorial (https://github.com/rumpkernel/wiki/wiki/Tutorial:-Building-Rumprun-Unikernels) I saw only compilation of single-file app. Now, with complex multi-libs app, I don't know what is build system supposed to do.

Well, in theory multilib applications are supposed to work exactly like no-lib applications, and actually quite a number of them do (see rumprun-packages). However, the toolchain is far from perfect. Also, C++ is haphazardly supported.

The first thing you'd need to do is to figure out where Foam::FatalError is supposed to be coming from. After that it's easier to say if it's a problem with the toolchain, C++ support, shared library side-effect, or what.

justinc1 commented 7 years ago

Shared libraries are not supported by Rumprun. Can you try to convince OpenFOAM to build/use static libraries?

I will try to. But when looking into some silver-bullet flag for wmake to produce .a libs, I got - 10 years old post, mentioning even older post, which contained partial instruction about how this could be done. I guess manual makefile hacking is unavoidable, in best case.

where Foam::FatalError is supposed to be coming from

I'm pretty sure it should be in some other shared lib. In that case, that lib should be converted to static-compiled one?

anttikantee commented 7 years ago

On 19/09/16 12:44, justinc1 wrote:

Shared libraries are not supported by Rumprun. Can you try to convince OpenFOAM to build/use static libraries?

I will try to. But when looking into some silver-bullet flag for wmake to produce .a libs, I got - 10 years old post, mentioning even older post, which contained partial instruction about how this could be done. I guess manual makefile hacking is unavoidable, in best case.

I'm not familiar with wmake, so I don't know. That said, the observed general problem with these "better" build systems is that they usually are not very generic, so it's quite probable you need to manually convince the build system to expand its horizons.

where Foam::FatalError is supposed to be coming from

I'm pretty sure it should be in some other shared lib. In that case, that lib should be converted to static-compiled one?

The point was to figure out for certain exactly which one, and to see if it's being linked in or not.

justinc1 commented 7 years ago

Ok, you can get static .a libs with 'wmake lib' (vs 'wmake libso', which is probably default). So that builds the OpenFOAM libs from ./src/ subdirectory. I got error from command like:

x86_64-rumprun-netbsd-ar x86_64-rumprun-netbsd-cr /path/to/libXYZ.a obj1.o obj2.o

"Fixed" it by editing x86_64-rumprun-netbsd-ar and replacing "x86_64-rumprun-netbsd-cr" with "-cr" - I guess that is what was intended. Now I have about 600 MB of .a libs in ./platforms/linux64GccDPOpt/lib.

Then comes the application/solver I'm interested into - the simpleFoam in particular.

cd applications/solvers/incompressible/simpleFoam/
wmake
file ../../../../platforms/linux64GccDPOpt/bin/simpleFoam 
  ELF 64-bit LSB relocatable, x86-64, version 1 (GNU/Linux), not stripped

The relocatable type should be OK, I think. But rumprun-bake hw_generic simpleFoam.bin simpleFoam complains:

/tmp/rumprun-bake.vUNjrh/tmp1.obj: In function `Foam::tmp<Foam::Field<Foam::Vector<double> > >::~tmp() [clone .part.54]':
simpleFoam.C:(.text+0x1a): undefined reference to `operator delete[](void*)'
/tmp/rumprun-bake.vUNjrh/tmp1.obj: In function `Foam::tmp<Foam::Field<double> >::~tmp() [clone .part.78]':
simpleFoam.C:(.text+0x2da): undefined reference to `operator delete[](void*)'
/tmp/rumprun-bake.vUNjrh/tmp1.obj: In function `Foam::tmp<Foam::Field<double> >::clear() const [clone .isra.80] [clone .part.81]':
simpleFoam.C:(.text+0x39b): undefined reference to `operator delete[](void*)'
simpleFoam.C:(.text+0x3a3): undefined reference to `operator delete(void*)'
/tmp/rumprun-bake.vUNjrh/tmp1.obj: In function `Foam::List<Foam::Vector<double> >::List(int, Foam::Vector<double> const&) [clone .constprop.442]':
simpleFoam.C:(.text+0x778): undefined reference to `operator new[](unsigned long)'
simpleFoam.C:(.text+0x81d): undefined reference to `__cxa_throw_bad_array_new_length'
...

nm simpleFoam  | grep ' U ' 
                 U abort
                 U atof
                 U __cxa_allocate_exception
                 U __cxa_atexit
                 U __cxa_bad_cast
                 U __cxa_begin_catch
                 U __cxa_call_unexpected
                 U __cxa_end_catch
                 U __cxa_free_exception
                 U __cxa_guard_acquire
                 U __cxa_guard_release
                 U __cxa_pure_virtual
                 U __cxa_throw
                 U __cxa_throw_bad_array_new_length
                 U __dso_handle
                 U __dynamic_cast
                 U __errno
                 U exit
                 U fclose
...
                 U _Unwind_Resume
                 U _ZdaPv
                 U _ZdlPv
                 U _ZN4Foam11fileMonitor11removeWatchEi
                 U _ZN4Foam11fileMonitor13setUnmodifiedEi
                 U _ZN4Foam11fileMonitor8addWatchERKNS_8fileNameE
                 U _ZN4Foam11fileMonitorC1Eb
                 U _ZN4Foam11fileMonitorD1Ev
                 U _ZN4Foam11findEtcFileERKNS_8fileNameEb
                 U _ZN4Foam11sigWriteNowC1EbRNS_4TimeE
                 U _ZN4Foam11sigWriteNowC1Ev
                 U _ZN4Foam11sigWriteNowD1Ev
                 U _ZN4Foam12findEtcFilesERKNS_8fileNameEbb

The first part should not be a problem, I hope - std lib, will be provided by rump. But I should try to include more libs to get also ZN4Foam11fileMonitor11removeWatchEi etc.

anttikantee commented 7 years ago

One thing you must do is teach the Makefiles to use ...-g++, not -gcc.

justinc1 commented 7 years ago

How do you know -gcc is used (yes, it is, but still ... :)? I thought fopen/fclose etc when used from C++ program will appear as "U fopen" etc in nm-output, just as above; and that there is nothing wrong with that.

(and thank you for the help)

anttikantee commented 7 years ago

On 19/09/16 15:20, justinc1 wrote:

How do you know -gcc is used (yes, it is, but still ... :)?

1) you included the compilation snippet earlier 2) the missing __cxa stuff is a telltale error

I thought fopen/fclose etc when used from C++ program will appear as "U fopen" etc in nm-output, just as above; and that there is nothing wrong with that.

Is this a problem or a general thinking-aloud statement? I don't understand it either way.

justinc1 commented 7 years ago

Aha, so the __cxa is the pointer.

The fopen comment was just thinking-aloud (I didn't know what is showing that -gcc is used instead of -c++).

justinc1 commented 7 years ago

Hm, I see:

cd ./src
./Allwmake lib
...
SOURCE=fileStat.C ;  x86_64-rumprun-netbsd-g++ -m64 -Dlinux64 -DWM_DP -Wall -Wextra -Wno-unused-parameter -Wold-style-cast -Wnon-virtual-dtor -O3  -DNoRepository -ftemplate-depth-100 -DFOAM_USE_INOTIFY -IlnInclude -I. -I/home/justin_cinkelj/devel/mikelangelo/rumprun-packages/openfoam/build//OpenFOAM-2.4.0/src/OpenFOAM/lnInclude -I/home/justin_cinkelj/devel/mikelangelo/rumprun-packages/openfoam/build//OpenFOAM-2.4.0/src/OSspecific/POSIX/lnInclude   -fPIC -c $SOURCE -o Make/linux64GccDPOpt/fileStat.o
fileStat.C:32:27: fatal error: sys/sysmacros.h: No such file or directory
compilation terminated.
fileStat.dep:107: recipe for target 'Make/linux64GccDPOpt/fileStat.o' failed

Is rump supposed to use heades from /usr/include (part of glibc-headers)?

anttikantee commented 7 years ago

No, glibc is not provided by Rumprun. You'll have to fix fileStat.C to not include the glibc-specific sys/sysmacros.h.