search

SPECFEM / specfem2d

SPECFEM2D simulates forward and adjoint seismic wave propagation in two-dimensional acoustic, (an)elastic, poroelastic or coupled acoustic-(an)elastic-poroelastic media, with Convolution PML absorbing conditions.
https://specfem.org
GNU General Public License v3.0
203 stars 147 forks source link

🐛 MPI partitioning of PML layers causes problems for externally generated mesh #1190

Open brennanbrunsvik opened 1 year ago

brennanbrunsvik commented 1 year ago

Description

I have a mesh generated in Cubit with PML layers. If I use ~4 cores, the waveform looks fine (with absorbing conditions turned on or off). When using more than ~6 cores, the waveforms become weird at the PMLs. The partitioning seems to cause long strips in the PMLs, some partitions cause waveforms that look reflecting, and some partitions look absorbing. These artifacts even appear if I turn off absorbing conditions while leaving my PML elements in place, so I wonder if this is specifically a partitioning and MPI communication issue.

I uploaded screenshots of a simplified case: homogenous elastic material, absorbing conditions turned off. These and the partitioner pdf file clearly show the issue. I also uploaded the par, source, stations, and mesh files needed to run this.

The wavefield before becoming problematic The wavefield after becoming problematic forward_image000003750 forward_image000004250 mesh_partitioning.pdf

specfem_issue_partitioning_pmls.zip

Affected SPECFEM2D version

dev branch, commit cea5867

Your software and hardware environment

Mac desktop. I don't have access right now to MPI and compiler numbers, but will add if asked.

Reproduction steps

I uploaded files that can reproduce the issue. Using the uploaded files, run meshfem2D and xspecfem2D from the unzipped folder.

Screenshots

No response

Logs

No response

OS

Mac

AbolfazlKhanMo commented 1 year ago

Hello,

Please check this issue #1174 out and see if you are using the right commands in the correct order when you want to create your model in CUBIT. I hope this helps!

Best, Abolfazl