Open efirvida opened 3 years ago
I can reproduce this with OpenFOAM v2012 (installed from .deb
) on Ubuntu 21.04, with preCICE v2.2.1 (built from source). My system has only two physical cores, and I use export OMPI_MCA_rmaps_base_oversubscribe=1
in my ~/.bashrc
.
It does not seem to matter if the interface is "cut" by the parallel boundary:
Since people have used the OpenFOAM adapter with more ranks and since we have also ran e.g. the turek-hron-fsi3
case with 25 ranks, this should be specific to the tutorial or the system.
@efirvida how many physical & logical cores do you have on your system?
@MakisH I'm running on a laptop with a i7-8650U, so I have 4 cores with 2 threads each, I test the old version of the tutorial rolling back the repository to the commit 5f4031fc7e45807dca787a525569b39a1909d2a3, and it works fine. I use the -oversubscribe
too and testit up to 12 partitions, I haven't time to compare the tutorials to see what's different, and also I haven't much experience with preCICE yet, but the old version didn't fail on any of my tests.
I think the crucial factor here is whether the master rank of OpenFOAM owns interface nodes or not. IIRC I had already a similar issue in the past. I'm still a bit puzzled whether the issue is triggered from the OpenFOAM side or from the preCICE side. I have some cases to test.. a workaround should still be given by this approach .
I think it is an issue in the adapter rather than preCICE. Some corner cases with empty master ranks were fixed in the preCICE bugfix release v2.2.1. and IIRC I already ran empty master cases with other solver.
I need to build the adapter in debug mode (CXX_FLAG='-g'
) to get more information here:
[2] #3 ? at Interface.C:?
[0] #4 preciceAdapter::Interface::Interface(precice::SolverInterface&, Foam::fvMesh const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, bool) at Interface.C:?
[2] #4 preciceAdapter::Interface::Interface(precice::SolverInterface&, Foam::fvMesh const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, bool) at ??:?
[0] #5 preciceAdapter::Adapter::configure() at ??:?
[2] #5 preciceAdapter::Adapter::configure() at ??:?
[0] #6 Foam::functionObjects::preciceAdapterFunctionObject::read(Foam::dictionary const&) at ??:?
[2] #6 Foam::functionObjects::preciceAdapterFunctionObject::read(Foam::dictionary const&) at ??:?
[0] #7 Foam::functionObjects::preciceAdapterFunctionObject::preciceAdapterFunctionObject(Foam::word const&, Foam::Time const&, Foam::dictionary const&) at ??:?
[2] #7 Foam::functionObjects::preciceAdapterFunctionObject::preciceAdapterFunctionObject(Foam::word const&, Foam::Time const&, Foam::dictionary const&) at ??:?
[0] #8 Foam::functionObject::adddictionaryConstructorToTable<Foam::functionObjects::preciceAdapterFunctionObject>::New(Foam::word const&, Foam::Time const&, Foam::dictionary const&) at ??:?
[2] #8 Foam::functionObject::adddictionaryConstructorToTable<Foam::functionObjects::preciceAdapterFunctionObject>::New(Foam::word const&, Foam::Time const&, Foam::dictionary const&) at ??:?
[0] #9 Foam::functionObject::New(Foam::word const&, Foam::Time const&, Foam::dictionary const&) at ??:?
[2] #9 Foam::functionObject::New(Foam::word const&, Foam::Time const&, Foam::dictionary const&) at ??:?
[0] #10 Foam::functionObjectList::read() at ??:?
[2] #10 Foam::functionObjectList::read() at ??:?
[0] #11 Foam::Time::run() const at ??:?
[2] #11 Foam::Time::run() const at ??:?
I can confirm that I can successfully run test cases (no OpenFOAM) where the master rank is not located at the interface.
Hi, I'm trying to run this tutorial in parallel just using `run.sh -parallel but always fail. I tried several configurations of decomposeParDict until I found that it only runs in parallel if I use this setting....
I'm running the preCICE adapters built with EasyBuild and the easyconfigs that I have made see it here: https://github.com/efirvida/easybuild-easyconfigs/commit/62611dc79313063019bce90ba83f42081c1fd998, So I'm really don't know if I have a mistake in my easyconfigs or is a tutorial error.
I have plans to submit the easyconfigs to the main EasyBuild repo but to do it I have to be sure that they work, and then follow my research on FSI.
Another thing that may be important to say is that I'm using Fedora 34 and I have some problems building the foss-2020a toolchain due to Binutils 2.34 bug (https://bugzilla.redhat.com/show_bug.cgi?id=1916925) and I change the version of the Binutils to 2.36.1 and Bison to 3.7.6 to the whole toolchain, and that's the main reason of my branch here https://github.com/efirvida/easybuild-easyconfigs/tree/fsi, I don't know if this introduces some bugs to the library.