search

FluidityProject / fluidity

Fluidity
http://fluidity-project.org
Other
362 stars 113 forks source link

Updates for Impish support #338

Closed tmbgreaves closed 2 years ago

tmbgreaves commented 2 years ago

This PR opens testing on Impish. Although we don't intend to offer formal Fluidity support for Impish, it's a useful starting point for early development on 22.04 LTS, which Fluidity should support, and which should be roughly the same as Impish at first.

tmbgreaves commented 2 years ago

@stephankramer - I wonder if you might be able to take a look at the Impish build, which I've set up with PETSc 3.15 built from the Debian Experimental package, aiming to avoid the issues you reported in 3.14. I'm seeing what I think is a PETSc-related build failure at the moment:

https://github.com/FluidityProject/fluidity/runs/3913986145?check_suite_focus=true

This is also using the gcc11 toolchain from Impish, aiming to get a first look at what will be landing in 22.04.

Thank you! :-)

Build environment available with:

docker run -it fluidity/baseimages:impish bash -il

And the Docker build for that failure is via the Dockerfile at:

https://github.com/FluidityProject/fluidity/blob/impish-updates/docker/actions/Dockerfile.actions.impish

tmbgreaves commented 2 years ago

Thank you very much @stephankramer :-) That's got through to linking now and reporting issues with Zoltan linking which I'll hack at when I get some time.

Patol75 commented 2 years ago

I tried to have a look, but, admittedly, I do not understand Fortran linker errors. Traceback available here.

angus-g commented 2 years ago

Looks like the configure might be picking up the wrong Zoltan. In the flags for the failing command, there is -ltrilinos_zoltan, but this doesn't reference the Zoltan installed through Fluidity's libzoltan package (available just as -lzoltan). Within the Docker image:

fluidity@c49083418931:~$ cat test.f90 
program test
use zoltan
use mpi
implicit none
type(zoltan_struct), pointer :: zz
zz => Zoltan_Create(MPI_COMM_WORLD)
print *, 'hello world'
end program

# successful link with the right flag
fluidity@c49083418931:~$ mpif90 test.f90 -I/usr/include -lzoltan -lptscotch -lmetis -lparmetis

# unsuccessful link with the flag picked up in the configure
fluidity@c49083418931:~$ mpif90 test.f90 -I/usr/include -ltrilinos_zoltan -lptscotch -lmetis -lparmetis
/usr/bin/ld: /tmp/cc2D80zI.o: in function `MAIN__':
test.f90:(.text+0x16): undefined reference to `__zoltan_MOD_zf90_create'
collect2: error: ld returned 1 exit status
Patol75 commented 2 years ago

Thanks @angus-g!

From what I can see, we have checking zoltan automagic... yes, which means that this bloc of code is never executed. Is it worth trying to force the execution of the latter or should the bloc of code above be changed?

Looking at other Actions workflows I had running recently, all of them had checking zoltan automagic... no. Therefore, I forced manual configuration of zoltan, and the build passed.

stephankramer commented 2 years ago

Yeah the issue is that we have always packaged zoltan for ourselves (see libzoltan and libzoltan-dev in https://launchpad.net/~fluidity-core/+archive/ubuntu/ppa/+packages) - I think the main reason being that the official Debian/Ubuntu packages (now built as part of the entire trilinos project: libtrilinos-zoltan-dev and libtrilinos12-dev) are not built with the fortran interface, so it does not have a zoltan.mod file. What's changed is that now also the official Debian/Ubuntu petsc package is linked with that zoltan from libtrilinos-zoltan12, and that it therefore adds a -ltrilinos_zoltan and therefore at the linking stage it picks up that library instead of the one from our own libzoltan(-dev).

I suspect trying to link to two different libzoltan's - our own that we call directly from fluidity and linked with -lzoltan and simultaneously also link with petsc and thus also with -ltrilinos_zoltan - even if possible, is just going to create havoc and be extremely sensitive to link line ordering.

So the two alternatives are:

  1. we adapt our own version of the libpetsc package to link to libzoltan(-dev) instead of libtrilinos. Here I wonder more generally why we are still carrying our own petsc packages in the ppa and what the difference is from the official Debian/Ubuntu ones: historically this was I think because we required optional external dependencies (mumps/superlu), required the fortran interface switched, and also possibly linking issues with zoltan. I think other than the zoltan issue, these have all gone away (the official packages are built with all we need including a petsc.mod fortran module), so I think we should consider to just use the official petsc packages instead and
  2. rebuilt libtrilinos-zoltan with fortran support. I think that if all we change is to request a .f90 fortran interface, the c interface should remain the same and we wouldn't need to recompile the petsc packages that depend on it. Unfortunately rebuilding libtrilinos is a bit of a pig as it requires rebuilding all of trilinos

I'll have a go at option 2. as I think it would be good if indeed we could drop the need for our own petsc packages - we would then only need a rebuild with hopefully very minor modification of the libtrilinos zoltan package.

stephankramer commented 2 years ago

This is ready to review now: all tests are passing on Impish

Main changes:

The main snag is that I found out that despite what it says in the commited baseimages for Bionic and Focal, we are not actually using gmsh 4.8 yet in the testing, instead the baseimages on Dockerhub (which the tests are run in) are still at a6d6bab770326ce8294ee48c5036fa3fff472f66 - meaning that all tests are currently run using gmsh 2.6

Upgrading to a downloaded gmsh 4.8/4.9 (or the 4.7 package that is available in Impish) obviously upsets quite a few tests that are grid-sensitive so I propose we deal with this in a separate PR