search

crotwell / sod

SOD - Standing Order for Data
GNU General Public License v2.0
13 stars 2 forks source link

Return " Error 400: Bad Request " when request data from GFZ instead of IRIS #2

Closed CovMat closed 6 years ago

CovMat commented 6 years ago

A minimal recipe to reproduce the bug:

<?xml version="1.0"?>
<sod>

<eventArm>
    <periodicFakeEventSource>
        <startTime>20111101T00:00:00Z</startTime>
        <interval>
            <unit>MINUTE</unit>
            <value>1</value>
        </interval>
        <numEvents>1</numEvents>
    </periodicFakeEventSource>
    <CSVEventPrinter/>
</eventArm>

<networkArm>
    <fdsnStation>
    <host>geofon.gfz-potsdam.de</host>
    <networkCode>GE</networkCode>
    <stationCode>BKNI</stationCode>
    <channelCode>BHZ</channelCode>
    </fdsnStation>
</networkArm>
</sod>

sod version is 3.2.8

If I run the recipe, it fails and return Error 400: Bad Request.

In the Sod.log, we can find following message:

2017-12-05 21:20:27,864 ERROR NetworkArm sod.Start - Arm NetworkArm failed: The remote web service just indicated that the query was badly formed. This may be because it does not support all of the parameters that SOD uses or it could be a bug in SOD. Check your recipe and if you cannot figure it out contact the developers at sod@seis.sc.edu.  Error: Code: 400 Error 400: Bad Request

simple time and window time parameters may not be combined

Usage details are available from /fdsnws/station/1/

Request:
/fdsnws/station/1/query?channel=BHZ&endafter=2011-11-01T00:00:00.000&includeavailability=false&includerestricted=false&level=station&network=GE&startbefore=2011-11-01T00:00:00.000&starttime=1993-01-01T00:00:01.000&station=BKNI

Request Submitted:
2017-12-05T13:20:27.623683

Service Version:
1.1.1
 on http://geofon.gfz-potsdam.de:80/fdsnws/station/1/query?channel=BHZ&endafter=2011-11-01T00:00:00.000&includeavailability=false&includerestricted=false&level=station&network=GE&startbefore=2011-11-01T00:00:00.000&starttime=1993-01-01T00:00:01.000&station=BKNI Sod is giving up and quiting.

If I copy the url http://geofon.gfz-potsdam.de:80/fdsnws/station/1/query?channel=BHZ&endafter=2011-11-01T00:00:00.000&includeavailability=false&includerestricted=false&level=station&network=GE&startbefore=2011-11-01T00:00:00.000&starttime=1993-01-01T00:00:01.000&station=BKNI in browser, GFZ returns:

Error 400: Bad Request

simple time and window time parameters may not be combined

However, If we delete starttime=1993-01-01T00:00:01.000 in the url, using: url http://geofon.gfz-potsdam.de:80/fdsnws/station/1/query?channel=BHZ&endafter=2011-11-01T00:00:00.000&includeavailability=false&includerestricted=false&level=station&network=GE&startbefore=2011-11-01T00:00:00.000&station=BKNI the correct XML file could be returned.

I think it may be a bug of SOD, could you please fix it?

crotwell commented 6 years ago

According to the fdsn spec, I believe that the server is wrong here. However, it is probaby easier for me to patch SOD to work around this than to get geofon to update their service.

crotwell commented 6 years ago

A prerelease with this fix is here: http://www.seis.sc.edu/downloads/sod/prerelease/3.2.9-SNAPSHOT/sod-3.2.9-SNAPSHOT3.tgz

In addition, the geofon server does not support includeAvailability, so you will need to add

false

to your element.

Please test and let me know if this fixes it for you.

CovMat commented 6 years ago

@crotwell

Thanks. I find another problem when I use version 3.2.9

I try to request data using following recipe but fail:

<?xml version="1.0"?> 
<sod> 

<eventArm> 
    <periodicFakeEventSource> 
        <startTime>20111101T00:00:00Z</startTime> 
        <interval> 
            <unit>MINUTE</unit> 
            <value>1</value> 
        </interval> 
        <numEvents>1</numEvents> 
    </periodicFakeEventSource> 
    <CSVEventPrinter/> 
</eventArm> 

<networkArm> 
    <fdsnStation> 
    <host>geofon.gfz-potsdam.de</host> 
    <includeAvailability>false</includeAvailability> 
    <networkCode>GE</networkCode> 
    <stationCode>PMG</stationCode> 
    <channelCode>BHZ</channelCode> 
    </fdsnStation> 
</networkArm> 

<waveformArm> 
    <bestChannelAtStation/> 
    <originOffsetRequest> 
        <beginOffset> 
            <unit>MINUTE</unit> 
            <value>0</value> 
        </beginOffset> 
        <endOffset> 
            <unit>MINUTE</unit> 
            <value>1</value> 
        </endOffset> 
        </originOffsetRequest> 
    <fdsnDataSelect/> 
    <someCoverage/> 
    <gapFill> 
            <zeroFill/> 
        </gapFill> 
</waveformArm> 

</sod>

In Fail.log we can find:

2017-12-06 10:08:54,210 INFO Refresh NetworkArm Fail.NetworkArm - 
GE.1993-01-01T00:00:00.000Z.PMG.1997-08-19T00:00:00.000Z
was rejected because the station was not active during the time range of requested events:  
StationEffectiveTimeOverlap:FAIL: 

2017-12-06 10:08:54,211 INFO Refresh NetworkArm Fail.NetworkArm - 
GE.1993-01-01T00:00:00.000Z.PMG.2009-08-09T00:00:00.000Z 
was rejected because the station was not active during the time range of requested events:  
StationEffectiveTimeOverlap:FAIL: 

2017-12-06 10:08:54,218 INFO Refresh NetworkArm Fail.NetworkArm - 
GE.1993-01-01T00:00:00.000Z.PMG.1993-09-10T00:00:00.000Z 
was rejected because the station was not active during the time range of requested events:  
StationEffectiveTimeOverlap:FAIL: 

2017-12-06 10:08:56,261 INFO Refresh NetworkArm Fail.NetworkArm - 
GE.1993-01-01T00:00:00.000Z.PMG..BHZ.1997-08-19T00:00:00.000Z 
was rejected because the channel was not active during the time range of requested events:  
ChannelEffectiveTimeOverlap:FAIL: 

2017-12-06 10:08:56,262 INFO Refresh NetworkArm Fail.NetworkArm - 
GE.1993-01-01T00:00:00.000Z.PMG.10.BHZ.2003-06-11T00:00:00.000Z 
was rejected because the channel was not active during the time range of requested events:  
ChannelEffectiveTimeOverlap:FAIL: 

2017-12-06 10:08:56,263 INFO Refresh NetworkArm Fail.NetworkArm - 
GE.1993-01-01T00:00:00.000Z.PMG.00.BHZ.1999-05-13T02:50:00.000Z 
was rejected because the channel was not active during the time range of requested events:  
ChannelEffectiveTimeOverlap:FAIL: 

2017-12-06 10:08:56,264 INFO Refresh NetworkArm Fail.NetworkArm - 
GE.1993-01-01T00:00:00.000Z.PMG.10.BHZ.2009-08-09T00:00:00.000Z 
was rejected because the channel was not active during the time range of requested events:  
ChannelEffectiveTimeOverlap:FAIL: 

2017-12-06 10:08:56,265 INFO Refresh NetworkArm Fail.NetworkArm - 
GE.1993-01-01T00:00:00.000Z.PMG.00.BHZ.2009-08-09T00:00:00.000Z 
was rejected because the channel was not active during the time range of requested events:  
ChannelEffectiveTimeOverlap:FAIL: 

2017-12-06 10:08:56,266 INFO Refresh NetworkArm Fail.NetworkArm - 
GE.1993-01-01T00:00:00.000Z.PMG.10.BHZ.2010-04-16T00:00:00.000Z 
was rejected because the channel was not active during the time range of requested events:  
ChannelEffectiveTimeOverlap:FAIL: 

2017-12-06 10:08:56,277 INFO Refresh NetworkArm Fail.NetworkArm - 
GE.1993-01-01T00:00:00.000Z.PMG.00.BHZ.2010-04-16T00:00:00.000Z 
was rejected because the channel was not active during the time range of requested events:  
ChannelEffectiveTimeOverlap:FAIL: 

2017-12-06 10:08:56,286 INFO Refresh NetworkArm Fail.NetworkArm - 
GE.1993-01-01T00:00:00.000Z.PMG..BHZ.1993-09-10T00:00:00.000Z 
was rejected because the channel was not active during the time range of requested events:  
ChannelEffectiveTimeOverlap:FAIL: 

2017-12-06 10:08:59,369 INFO WaveformArm 1 Fail.Waveform - EventChannelPair: 
(0) Event: Unknown (-1) | 11/01/2011 00:00:00 GMT | Mag: -10.0 | Depth 0.00  
km | (0.0, 0.0) GE.PMG.10.BHZ Event channel subsetter rejected:  
BestChannelAtStation:FAIL: 

2017-12-06 10:08:59,992 INFO WaveformArm 1 Fail.Waveform - EventChannelPair: 
(1) Event: Unknown (-1) | 11/01/2011 00:00:00 GMT | Mag: -10.0 | Depth 0.00  
km | (0.0, 0.0) GE.PMG.00.BHZ Processor rejected SomeCoverage:FAIL:0 filters  
returned by available data

However, if we check the XML file in GFZ: http://geofon.gfz-potsdam.de/fdsnws/station/1/query?channel=BHZ&endtime=2011-11-01T00:00:00.000&includeavailability=false&includerestricted=false&level=channel&network=GE&station=PMG , this station still operate after 11/01/2011 00:00:00 GMT (marked by red arrows in following figure) 1

And I can also download data from GFZ web service (http://eida.gfz-potsdam.de/webdc3/) during this time window.

I think it may be another problem of GFZ server, but could you please patch SOD to work around it? Thank you very much!

crotwell commented 6 years ago

You are getting the station metadata from geofon, but if you look in the Sod.log file you will see you are asking IRIS for the seismograms. If you want to get them from geofon, you need to set the host within the as well.

Also, you may want to add this to skip the available data check as they do not support it.

<sod>
<properties>
  <skipAvailableData>TRUE</skipAvailableData>
</fdsnDataSelect>

You might also want to change someCoverage to someDataCoverage so it works on teh seismograms.

And of course at the bottom of the waveformArm you probably want a sacWriter or a miniseedWriter to save the seismogram to file.