Trouble porting model from 1 to multiple mpi ranks

underworldcode / underworld2

underworld2: A parallel, particle-in-cell, finite element code for Geodynamics.

http://www.underworldcode.org/

Other

168 stars 58 forks source link

Trouble porting model from 1 to multiple mpi ranks #664

Open gduclaux opened 1 year ago

gduclaux commented 1 year ago

G'day UW folks!

I have an issue with porting an UW model from serial to parallel and I was wondering if you could kindly give me a hand? I’ve setup a Tapponnier-like indenter model that runs fine in Docker directly from the notebook but rises error in the model.solve() in parallel (I tried running on 2 and 4 mpi ranks in Docker terminal).

MontagePlasticStrain

Hope one of you can point me the issue...

Cheers

Guillaume

Tapponnier_extrusionAsia-final.ipynb.zip

julesghub commented 1 year ago

Hi Gilly,

Good to hear from you. Nice model! The problem is to do with the multigrid solver and the indenter (internal BC). I'm working on a fix now. To get around the problem you can use the mumps solver. i.e.

Model.solver.set_inner_method("mumps")

For 2D models this works well and gets around the problem. For 3D solves it DOESN"T work well because mumps is a direct solve method that consumes far too much memory. Let me know how it goes.

cheers, Jules

gduclaux commented 1 year ago

Thanks Jules! Sorry for the delayed response. I'll give it a go and let you know if I have any issue with this workaround.

Cheers, Gilly

gduclaux commented 1 year ago

Quick feedback on the workaround you pointed out:

the model starts on multiple ranks with mumps (Model.solver.set_inner_method("mumps")) but it takes forever to converge... I tried adding some penalty too, but it doesn't do any good.
following your original suggestion Jules I did play around a bit and found that using superludist for the inner solves (Model.solver.set_inner_method("superludist")) is doing a better job with this setup.

Thanks again for your help Jules!