SPECFEM / specfem3d

SPECFEM3D_Cartesian simulates acoustic (fluid), elastic (solid), coupled acoustic/elastic, poroelastic or seismic wave propagation in any type of conforming mesh of hexahedra (structured or not).
GNU General Public License v3.0
403 stars 225 forks source link

Small spectral elements #1501

Closed eikmeier-cn closed 2 years ago

eikmeier-cn commented 2 years ago

Hey guys.

I am trying to perform a simulation in a small volume (450 x 400 x 90 m) with two layers. The first layer is soil with low velocity values and the second layer is a gneiss. For the source I will use the injection of a signal with high frequencies (40Hz). So, my spectral elements must be small. For the first layer the size is 1.25m and for the second layer 11.25m. However, when I try to run the simulation, the following error appears:

print_do_erro - Copia

When I test the same scenario increasing the volume (40000 x 45000 x 9000m) and using 300m spectral elements for the first layer and 900m for the second layer, the same error does not happen.

The script that I used to create the first and the second case is the same, and for that I used Cubit.

Could anyone help me with this issue?

Thank you!

eikmeier-cn commented 2 years ago

I solved the problem. We have to increase the value of the TOLERANCE_FACE_DETECTIONparameter, in the constants.h.in file, from 1d-4 to 1d2. Doing this still occurs one more problem, a floating point error. To solve this we have to change the CUSTOM_REAL parameter from @CUSTOM_REAL@ to SIZE_DOUBLE in the constants.h.in and the CUSTOM_MPI_TYPE parameter from @CUSTOM_MPI_TYPE@ to MPI_DOUBLE_PRECISION in the precision.h.in. After that everything works fine. Don't forget to recompile the code again (./configure... and then make...)

However, I changed the TOLERANCE_FACE_DETECTION parameter without knowing its exact meaning. If someone from the Specfem team can briefly clarify the meaning of this parameter, that would be great. Thanks!

eikmeier-cn commented 2 years ago

Well, I said I had solved the problem, but another one cames up. A floating point error in the solver. Even setting all possible variables to double precision in constants.h.in and compiling the code, I couldn't solve this problem. Can someone from the Specfem team helpe me with this?

image

eikmeier-cn commented 2 years ago

Problem solved. The problem was in the DT. The rule exposed in the manual can fail in several situations. So we need to decrease the dt until it works.

danielpeter commented 2 years ago

when looking at the face detection error, I'm still a bit puzzled why it didn't work. other than that the indexing on the corners is permuted, the corner locations seem to fit well. unfortunately, without your mesh file it's hard to debug that. for example, not sure if a cubit command like compress all sort would help to avoid this.

the parameter TOLERANCE_FACE_DETECTION is a factor used to identify the midpoint of a given element face. the distance between the target midpoint and all face midpoints is then compared to this factor times the edge length of the element. since this factor multiplies with the element edge length, it usually avoids problems if the mesh elements are large or small.

regarding choosing a sufficiently small DT, the output_generate_database.txt file contains some output info to suggest a maximum DT size based on some heuristic CFL condition values, like:

...
*** Maximum suggested time step =    4.86940965E-02
...
eikmeier-cn commented 2 years ago

Thanks for the feedback, @danielpeter.