SPECFEM / specfem3d

SPECFEM3D_Cartesian simulates acoustic (fluid), elastic (solid), coupled acoustic/elastic, poroelastic or seismic wave propagation in any type of conforming mesh of hexahedra (structured or not).
GNU General Public License v3.0
392 stars 223 forks source link

Source in acoustic layer with an external source time function file no longer works #1522

Closed ammcpherson closed 1 year ago

ammcpherson commented 1 year ago

Hello,

In between this commit and the latest commit, there is no longer the capability to have a source in an acoustic layer and use an external source time function file. To briefly sum up:

For this commit

No External STF External STF
Source in Elastic Medium Works Works
Source in Acoustic Medium Works Works

For latest commit

No External STF External STF
Source in Elastic Medium Works Works
Source in Acoustic Medium Works Does Not Work

By works, I mean that the solver exits successfully and generates some sort of record. I have attached the DATA folder for a simple example built up from the homogeneous halfspace example that includes an elastic and an acoustic layer.

Thank you for your time, Amanda McPherson DATA.zip

carltape commented 1 year ago

Thanks, Amanda. It's probably worth adding that the objective is to reproduce the synthetics generated in this paper by Jordan Bishop. It looks like we can do it in the older code but not the latest.

danielpeter commented 1 year ago

hi Amanda, I would need some more input from your side here. after checking the external STF feature for sources in elastic and acoustic domains, the code seems to work properly - also with your mesh setup.

maybe there is some inconsistency in the simulation setup. in your DATA.zip attachment, it looks like the external STF provided by DATA/source_time_function.txt comes from a source in an elastic domain (i.e., a quasi-heaviside function).

however, if you run the simulation with a source in the acoustic domain, the STF changes to a default Gaussian (pressure like pulse sources). thus, if you want to use an external STF to reproduce these outputs, then the source_time_function.txt would need to change as well. please check the STF used by setting in Par_file:

PRINT_SOURCE_TIME_FUNCTION      = .true.

this produces the OUTPUT_FILES/plot_source_time_function.txt file from which you can easily construct the corresponding external STF for the source in the acoustic domain.

still, that wouldn't explain why in a previous commit things were working, unless there was a mix-up with the simulation setups. so, please let me know if I'm missing something.

ammcpherson commented 1 year ago

Hi Daniel,

I will provide an updated STATIONS and CMTSOLUTION file, which produces the source/receiver geometry shown in 'sr_real.png.' Here the source is definitely in the acoustic layer. The external source time function source_time_function.txt was created using a Gaussian function, and looks like 'est_plot.png.'

Also, as Carl said, the goal initially was to recreate published results, and the external source time function used there was also a Gaussian function. I could easily run the simulation, as long as I did not use the external source time function.

est_plot sr_real Updated.zip

danielpeter commented 1 year ago

hi Amanda, the updated zip-file misses the external source time function.

again, if I use the same external STF attached here as the internal Gaussian one (checked through OUTPUT_FILES/plot_source_time_function.txt), then the simulation outputs are the same (up to the origin time shift).

external_stf.txt

ammcpherson commented 1 year ago

Hi Daniel,

I'm sorry, I should have been more explicit: I did not add an updated external source time function, because the one in the original zip file is a Gaussian function. It was created to be the same as the internal one for this example, so as to compare the output from the basic homogeneous halfspace example. I just added the plot to double check that it was not a heavyside function.

I just checked my example with your attached external_stf.txt, and the solver does not work on my instance of the latest commit of code. It exits early with the same error as I have come to expect. I also just checked out a fresh install of SPECFEM and tried the same example with your external STF, and the solver exits early on the fresh install as well.

Sincerely, Amanda

danielpeter commented 1 year ago

strange, i still can't reproduce the error.

could you attach all your output_* files, the plot_source_time_function.txt and some sample trace (if available) for the run that fails?

carltape commented 1 year ago

Thanks, Daniel. At the moment, the goal is to see if you generate the same failed run as we do. I am starting with this clone: git clone --depth 1 --branch devel https://github.com/geodynamics/specfem3d specfem3d_ifort I am attaching the full input and output files here for the failed run that I get. DATA.tar.gz OUTPUT_FILES.tar.gz The mesher works and so does generating databases. Then the solver fails with an error like forrtl: error (65): floating invalid My run is using the external_stf.txt file that you attached earlier. (See bottom of the CMTSOLUTION file.) And I also set PRINT_SOURCE_TIME_FUNCTION = .true. as you were doing. The last line listed in output_solver.txt is this printing the source-time function yet no stf is written out. When the print flag is set to false, then the last line of output_solver.txt is Starting time iteration loop... The exit error is the same in both cases. It would be good to know if your run also fails when using the attached DATA files as input. Thanks again.

danielpeter commented 1 year ago

what Intel compiler were you using? let's try again with this update #1523

carltape commented 1 year ago

It runs! @ammcpherson can you please verify, too, and compare with the output from the older version of specfem? Daniel, we are using ifort (IFORT) 18.0.5 20180823 Our new cluster is intel 2022, but we haven't tested it there yet. Exciting to have these capabilities and setting us up for extending the work of Jordan Bishop and others. Thank you!

ammcpherson commented 1 year ago

Hi Daniel,

I can confirm that this example runs for me as well, and I successfully generate some synthetic seismograms using the latest update to the code.

Thank you very much for your help with this issue! -Amanda

gassmoeller commented 1 year ago

@ammcpherson it sounds like this issue has been resolved? If so, please close the issue so we know that it requires no further work. Thanks for the report.

ammcpherson commented 1 year ago

@gassmoeller We are hoping to validate some synthetics before we completely close the issue. In the next week, we should have the information we need to either close the issue or ask for further support.

Would it be better to open a new issue should our validation fail?

gassmoeller commented 1 year ago

No let's keep this open until you have the data and continue the discussion here, that way we have a single place for this issue. Just checking, because your last comment sounded like the issue was resolved.

ammcpherson commented 1 year ago

@jwbishop has detailed our efforts to reproduce figures from the aforementioned paper in this new issue. As this initial issue seems to have been related to differences in the compiler we use on our HPC and the compiler @danielpeter uses to develop, I will close this issue so we can focus on the new one.