search

krischer / instaseis

Instant high-frequency seismograms from an AxiSEM database
http://instaseis.net
Other
51 stars 23 forks source link

Inconsistent trace data from Syngine for the same request #76

Closed manoch-iris closed 4 years ago

manoch-iris commented 4 years ago

Hi,

Inconsistent data for the same request are returned by the Syngine Web Service when the source mechanism is set to sourceforce. The issue can be simulated outside the Syngine using;

db = instaseis.open_db()

In both cases, the minimum and maximum values of the returned traces change as the same request is submitted multiple times. To repeat the cycle, Syngine servers must be restarted or the open_db() statement above must be re-executed.

The Example 1 below shows a request that was submitted to Syngine 6 times with the min and max values of the traces returned by Syngine for each run. Our testing reveals that:

• This behavior is only observed for sourceforce. Other source mechanism ( sourcedoublecouple and sourcemomenttensor) appear to be functioning correctly. • This behavior is observed regardless of the model used.

The issue appears to be caused by the code segment that creates and populates the mesh.displ_buffer. In file : reciprocal_merged_instaseis_db.py by replacing the if block starting at line 387

if id_elem not in mesh.displ_buffer:
        utemp = self._get_and_reorder_utemp(id_elem)
        mesh.displ_buffer.add(id_elem, utemp)
else:
        utemp = mesh.displ_buffer.get(id_elem)

with:

utemp = self._get_and_reorder_utemp(id_elem)

The returned results are consistent (Example 2).

Example 1 (original code):

http://service.iris.washington.edu/iriswsbeta/syngine/1/query?format=miniseed&components=ZRT&units=displacement&model=ak135f_2s&origintime=2016-05-22T07:57:34.000&starttime=2016-05-22T07:57:24.000&endtime=2016-05-22T08:00:54.000&network=AK&station=HOM&sourcelatitude=60.0273&sourcelongitude=-153.0683&sourcedepthinmeters=0&sourceforce=-1e22,0,0&nodata=404

MODEL: ak135f_2s
SOURCE: sourceforce=-1e22,0,0

Loop 0:
2020-07-07 07:37:48.698765
BXZ min: -1300363.25, max: 925868.4375
BXR min: -879301.5, max: 563970.0
BXT min: -3.2123164805247795e-12, max: 2.7061482395229763e-12

Loop 1:
2020-07-07 07:37:49.340336
BXZ min: -1300903.625, max: 924770.8125
BXR min: -567468.4375, max: 879446.9375
BXT min: -3.2123164805247795e-12, max: 2.7061482395229763e-12

Loop 2:
2020-07-07 07:37:49.827122
BXZ min: -1300903.625, max: 924770.8125
BXR min: -8341.162109375, max: 5478.35400390625
BXT min: -3.2123164805247795e-12, max: 2.7061482395229763e-12

Loop 3:
2020-07-07 07:37:50.357499
BXZ min: -1300903.625, max: 924770.8125
BXR min: -8341.162109375, max: 5478.35400390625
BXT min: -3.2123164805247795e-12, max: 2.7061482395229763e-12

Loop 4:
2020-07-07 07:37:50.843558
BXZ min: -1300903.625, max: 924770.8125
BXR min: -8341.162109375, max: 5478.35400390625
BXT min: -3.2123164805247795e-12, max: 2.7061482395229763e-12

Loop 5:
2020-07-07 07:37:51.382799
BXZ min: -1300903.625, max: 924770.8125
BXR min: -8341.162109375, max: 5478.35400390625
BXT min: -3.2123164805247795e-12, max: 2.7061482395229763e-12

Example 2 (modified code):

http://service.iris.washington.edu/iriswsbeta/syngine/1/query?format=miniseed&components=ZRT&units=displacement&model=ak135f_2s&origintime=2016-05-22T07:57:34.000&starttime=2016-05-22T07:57:24.000&endtime=2016-05-22T08:00:54.000&network=AK&station=HOM&sourcelatitude=60.0273&sourcelongitude=-153.0683&sourcedepthinmeters=0&sourceforce=-1e22,0,0&nodata=404

MODEL: ak135f_2s
SOURCE: sourceforce=-1e22,0,0

Loop 0:
2020-07-07 08:58:37.983106
BXZ min: -1300363.25, max: 925868.4375
BXR min: -879301.5, max: 563970.0
BXT min: -3.2123164805247795e-12, max: 2.7061482395229763e-12

Loop 1:
2020-07-07 08:58:38.809414
BXZ min: -1300363.25, max: 925868.4375
BXR min: -879301.5, max: 563970.0
BXT min: -3.2123164805247795e-12, max: 2.7061482395229763e-12

Loop 2:
2020-07-07 08:58:39.301468
BXZ min: -1300363.25, max: 925868.4375
BXR min: -879301.5, max: 563970.0
BXT min: -3.2123164805247795e-12, max: 2.7061482395229763e-12

Loop 3:
2020-07-07 08:58:39.794067
BXZ min: -1300363.25, max: 925868.4375
BXR min: -879301.5, max: 563970.0
BXT min: -3.2123164805247795e-12, max: 2.7061482395229763e-12

Loop 4:
2020-07-07 08:58:40.283487
BXZ min: -1300363.25, max: 925868.4375
BXR min: -879301.5, max: 563970.0
BXT min: -3.2123164805247795e-12, max: 2.7061482395229763e-12

Loop 5:
2020-07-07 08:58:40.871748
BXZ min: -1300363.25, max: 925868.4375
BXR min: -879301.5, max: 563970.0
BXT min: -3.2123164805247795e-12, max: 2.7061482395229763e-12
chad-earthscope commented 4 years ago

This is a pretty serious problem for Syngine. It is unclear if our diagnostic fix of skipping an apparently re-used buffer will result in major performance issues or not. If not, we could apply a hot fixed until this is resolved. Any time to consider this @krischer ?

krischer commented 4 years ago

Yes - sorry - this has been on my backburner for a while.

It heavily depends on the access pattern of course but effectively turning off the buffer might have a very negative effect on performance and I/O load. I'm pretty sure that what happens is that the data is read from the buffer and somehow modified in-place so the buffered version changes . This of course should not happen and a well placed copy of the array would solve that.

As a quick fix with much less of a potential performance impact you could just copy immediately after it is retrieved from the buffer.

Otherwise I'll hopefully have some time to look at this early next week.

krischer commented 4 years ago

Hi all! A bit delayed from my side but I think I fixed the issue here: https://github.com/krischer/instaseis/pull/78

In the end it is a one liner with a (I think) very minimal performance impact but I also added a regression test. Can you verify that this fixes the issue on your side? I'd then cut a new release.

manoch-iris commented 4 years ago

Hi Lion,

Thank you for looking into this. We have verified that your change fixes the issue at our end.

krischer commented 4 years ago

Instaseis version 1.4.2 is now available on pypi so if you have a fortran compiler you can just run pip install -U instaseis and it should update. Conda packages might take a few more days - if this finishes soonish I can still merge it, otherwise I'll be off tomorrow and it will have to wait until the end of the week: https://github.com/conda-forge/instaseis-feedstock/pull/23

liamtoney commented 4 years ago

Thanks, Lion!

manoch-iris commented 4 years ago

Thank you Lion! I will wait for the conda package.

—manoch

On Aug 11, 2020, at 2:57 PM, Lion Krischer notifications@github.com wrote:

 Instaseis version 1.4.2 is now available on pypi so if you have a fortran compiler you can just run pip install -U instaseis and it should update. Conda packages might take a few more days - if this finishes soonish I can still merge it, otherwise I'll be off tomorrow and it will have to wait until the end of the week: conda-forge/instaseis-feedstock#23

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

krischer commented 4 years ago

The conda package is out now.

manoch-iris commented 4 years ago

Thank you Lion, I really appreciate it. Syngine is upgraded.

—manoch

On Aug 13, 2020, at 1:31 AM, Lion Krischer notifications@github.com wrote:

The conda package is out now.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/krischer/instaseis/issues/76#issuecomment-673341615, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABL5L2VWSFDYUDT7Y5YKWDTSAOQHRANCNFSM4OS7E3VA.