Update Makefile for parallel CalculiX simulation

uekerman commented 4 years ago

Modification of the Makefile of Calculix to make parallel runs possible with preCICE. Contributed by @FabienSalmon. How can we best integrate this?

3rav commented 3 years ago

I suggest to be more interested in the version based on PaStiX solver (https://gitlab.inria.fr/solverstack/pastix), the difference in performance is significant.

MakisH commented 2 years ago

@FabienSalmon Do I understand correctly that this change adds a feature of SPOOLES, which we should always want to use? Is there any situation in which this could cause any issues?

If this is something we can take as-is, then let's just merge it.

boris-martin commented 2 years ago

I checked documentation (http://www.dhondt.de/ccx_2.16.pdf, page 12) and, to avoid confusion: this change impacts the use of parallelism within Spooles: multi-threading for CalculiX' own code is enabled without USE_MT flag (I think -fopenmp is sufficient).

I managed to compile and link without the additional linking file. I just had the default makefile from master branch (which just links -lspooles), with the Ubuntu (Mint actually) version of Spooles and it worked (the solver prints "Using X cpus for spooles" and I got a moderate speedup)

Note for configuration: In the tutorial for partitioned beam, we have the following lines in the run script:

export OMP_NUM_THREADS=1
export CCX_NPROC_EQUATION_SOLVER=1

It is sufficient to define only OMP_NUM_THREADS to get the parrallelization, but the second flag takes precedence. So I ran with (2,1) and got no parallelism before realizing I had to ue (2,2) or just OMP_NUM_THREADS=2 without the second line.

Short version:

I'd add the -DUSE_MT flag on the makefile by default (maybe not on this PR since makefiles get changed a lot in the 4-5 PRs we're handling right now) and discard the other change.
We should indicate somewhere on the website that we recommend using an appropriate value of OMP_NUM_THREADS (and/or other flags) to get parallelization on a shared memory machine.
To clear some potential confusion: there is no support for distributed memory (MPI etc) parallelization within CalculiX, with or without this PR.

MakisH commented 2 years ago

Thanks a lot for figuring this out! Maybe open an issue in the website to document this later (if not directly tomorrow).

MakisH commented 2 years ago

@boris-martin will document directly on the website. We can close this now.

precice / calculix-adapter

Update Makefile for parallel CalculiX simulation #34