EIDA / userfeedback

This repository is meant to collect feedback from EIDA users by means of its Issue Tracker
11 stars 5 forks source link

French stations missing in _ALPARRAY virtual network? #36

Closed javiquinte closed 4 years ago

javiquinte commented 5 years ago

On behalf of @manconia

I now see the RESIF AlpArray temporary stations if I use net=Z3. However, if I call the virtual net=_ALPARRAY and try to get the .mseed files I actually get only data from 4 RESIF stations and none of the Z3...

Any idea?


From the AlpArray website:

Virtual Network Code _ALPARRAY In order to identify and facilitate easy and rapid access to all AlpArray Seismic Network stations, we have created a virtual network for AlpArray - _ALPARRAY. This network code can be used using both the EIDA data access portal and scripts using fdsn_fetch or arclink_fetch.

The rules defining which stations are included in _ALPARRAY are:

There are a number of exceptions that are also included (whitelisted):

Originally posted by @manconia in https://github.com/EIDA/userfeedback/issues/35#issuecomment-553134707

javiquinte commented 5 years ago

Hi @manconia ! Could you provide one or two examples of stations which you don't see in _ALPARRAY and actually should be there? And which are the 4 stations you do see?

sheimers commented 5 years ago

I have tried the following request and got some data from France including several stations from the FR and GU network, but no Z3 network stations from France.

My first guess was it could be an authentication problem because Z3 were missing, but Z3 can be downloaded if I select single stations or the Z3 network in stead of _ALPARRAY. My new guess is now that it fails for a specific station and then stops downloading. I'll do one more try later today and look at the error messages more carefully.

fdsnws_fetch -a eidatoken -s 2019-10-15T16:00:00Z -e 2019-10-15T16:05:00Z -N _ALPARRAY -v -o data.mseed

My error messages (including the missing french Z3 stations):

did not receive data from CH.OTER1.., CR.BRJN.., CR.HVAR.., CR.KALN.., CR.KIJV.., CR.MORI.., CR.MOSL.., CR.NVLJ.., CR.OZLJ.., CR.PLIT.., CR.PTJ.., CR.RABC.., CR.RIY.., CR.SMRN.., CR.UDBI.., CR.VIRC.., CZ.CKRC.., CZ.GOPC.., CZ.PBCC.., FR.CORF.., FR.FLAF.., FR.FOURG.., FR.MOF.., FR.OGGM.., FR.PIAF.., FR.PRIMA.., FR.VAL4.., FR.VDH4.., GR.GRA3.., GR.GRB4.., GR.MILB.., GR.TMO22.., GU.LSD.., HU.EGYH.., HU.MPLH.., IV.CRE.., IV.FSSB.., IV.MAGA.., IV.SALO.., MT.AVP.., NI.DST2.., NI.VINO.., OX.FERB.., RD.ETNF.., RD.LOR.., RD.ORIF.., RD.PGF.., SI.ABSI.., SK.VYHS.., SL.LEGS.., SL.ZALS.., ST.GAGG.., Z3.A010A.., Z3.A011B.., Z3.A017A.., Z3.A020B.., Z3.A036A.., Z3.A050A.., Z3.A051A.., Z3.A052A.., Z3.A071A.., Z3.A072A.., Z3.A073A.., Z3.A074A.., Z3.A075A.., Z3.A076A.., Z3.A077A.., Z3.A078A.., Z3.A079A.., Z3.A080A.., Z3.A081A.., Z3.A082A.., Z3.A083A.., Z3.A084A.., Z3.A085A.., Z3.A086A.., Z3.A087A.., Z3.A088B.., Z3.A089A.., Z3.A090A.., Z3.A104A.., Z3.A126A.., Z3.A127A.., Z3.A129A.., Z3.A153A.., Z3.A156A.., Z3.A157B.., Z3.A158A.., Z3.A159A.., Z3.A160A.., Z3.A162A.., Z3.A163A.., Z3.A165A.., Z3.A166B.., Z3.A167A.., Z3.A168A.., Z3.A169A.., Z3.A170A.., Z3.A171A.., Z3.A172B.., Z3.A173A.., Z3.A174A.., Z3.A175A.., Z3.A176A.., Z3.A177A.., Z3.A178B.., Z3.A179A.., Z3.A180A.., Z3.A181A.., Z3.A182A.., Z3.A183A.., Z3.A184A.., Z3.A185A.., Z3.A187A.., Z3.A188A.., Z3.A190A.., Z3.A191A.., Z3.A192B.., Z3.A194A.., Z3.A195A.., Z3.A196A.., Z3.A198A.., Z3.A204A.., Z3.A206A.., Z3.A208A.., Z3.A209B.., Z3.A210A.., Z3.A213A.., Z3.A215A.., Z3.A216A.., Z3.A250A.., Z3.A252A.., Z3.A253A.., Z3.A254A.., Z3.A266A.., Z3.A268A.., Z3.A269A.., Z3.A270A.., Z3.A280A.., Z3.A283B.., Z3.A284A.., Z3.A286A.., Z3.A288A.., Z3.A289A.., Z3.A316A.., Z3.A338A.., Z3.A351A.., ZS.D001.., ZS.D002.., ZS.D003.., ZS.D004.., ZS.D005.., ZS.D006.., ZS.D007.., ZS.D008.., ZS.D009.., ZS.D010.., ZS.D011A.., ZS.D012.., ZS.D013.., ZS.D014.., ZS.D015.., ZS.D016A.., ZS.D017.., ZS.D018.., ZS.D019.., ZS.D020.., ZS.D021.., ZS.D022.., ZS.D023.., ZS.D024.., ZS.D025.., ZS.D026.., ZS.D027.., ZS.D028.., ZS.D029.., ZS.D030.., ZS.D031.., ZS.D032.., ZS.D033.., ZS.D034.., ZS.D035.., ZS.D036.., ZS.D037.., ZS.D038.., ZS.D039.., ZS.D040.., ZS.D041.., ZS.D042.., ZS.D043.., ZS.D044.., ZS.D045A.., ZS.D046.., ZS.D047.., ZS.D048.., ZS.D049.., ZS.D050.., ZS.D051.., ZS.D052.., ZS.D053.., ZS.D054.., ZS.D055.., ZS.D056.., ZS.D057.., ZS.D058.., ZS.D059.., ZS.D060.., ZS.D061.., ZS.D062.., ZS.D063.., ZS.D064.., ZS.D065.., ZS.D066.., ZS.D067.., ZS.D068.., ZS.D069.., ZS.D070.., ZS.D071.., ZS.D072.., ZS.D073.., ZS.D074.., ZS.D075.., ZS.D076.., ZS.D077.., ZS.D078.., ZS.D079.., ZS.D081.., ZS.D082.., ZS.D083.., ZS.D084.., ZS.D085.., ZS.D086.., ZS.D087.., ZS.D088.., ZS.D089.., ZS.D090.., ZS.D091.., ZS.D092.., ZS.D093.., ZS.D094.., ZS.D095.., ZS.D096.., ZS.D097.., ZS.D098.., ZS.D099.., ZS.D100.., ZS.D101.., ZS.D102.., ZS.D103.., ZS.D104.., ZS.D105.., ZS.D106.., ZS.D107.., ZS.D108.., ZS.D109.., ZS.D110.., ZS.D111.., ZS.D112.., ZS.D113.., ZS.D114.., ZS.D115.., ZS.D116.., ZS.D117.., ZS.D118.., ZS.D119.., ZS.D120.., ZS.D122A.., ZS.D123.., ZS.D124.., ZS.D125.., ZS.D126.., ZS.D127.., ZS.D128.., ZS.D129.., ZS.D130.., ZS.D131.., ZS.D132.., ZS.D133.., ZS.D134.., ZS.D135.., ZS.D136.., ZS.D137.., ZS.D138.., ZS.D139.., ZS.D140.., ZS.D141.., ZS.D142.., ZS.D143.., ZS.D144.., ZS.D145.., ZS.D146.., ZS.D147.., ZS.D148.., ZS.D149.., ZS.D150.., ZS.D151.., ZS.D152.., ZS.D153.., ZS.D154.., ZS.D155.., ZS.D156.., ZS.D157.., ZS.D158.., ZS.D159.., ZS.D160.., ZS.D161.., ZS.D162.., ZS.D163..

sheimers commented 5 years ago

What I noticed: When I select _ALPARRAY and a specific french Z3 station it works. If I select Z3 and all stations I do get Z3 stations from France and from others. But If I select _ALPARRAY and don't specify stations I get Z3 stations from other countries, but none from France, but I do get some other french stations from FR and GU networks.

jschaeff commented 5 years ago

@javiquinte how is "_ALPARRAY" expended to a network code ? I can't see it in the fdsnws_fetcj source.

manconia commented 5 years ago

@javiquinte I experience the same as @sheimers.

manconia commented 4 years ago

@javiquinte

Update: this is the result I have now, which is even worst than before as I see much less stations...

fdsnws_fetch -a eidatoken -N _ALPARRAY -C HHZ -s 2017-08-23T07:29:00 -e 2017-08-23T07:29:10 -v -o 2017-08-23T07:29:00_2017-08-23T07:29:10.mseed


using token in eidatoken: [Errno 2] No such file or directory: 'gpg': 'gpg' getting routes from http://geofon.gfz-potsdam.de/eidaws/routing/1/query?network=_ALPARRAY&channel=HHZ&starttime=2017-08-23T07%3A29%3A00&endtime=2017-08-23T07%3A29%3A10&format=post did not receive routes to _ALPARRAY...HHZ authenticating at https://eida.ethz.ch/fdsnws/dataselect/1/auth authenticating at https://erde.geophysik.uni-muenchen.de/fdsnws/dataselect/1/auth authenticating at https://ws.resif.fr/fdsnws/dataselect/1/auth authentication at https://eida.ethz.ch/fdsnws/dataselect/1/auth successful getting data from http://eida.ethz.ch/fdsnws/dataselect/1/queryauth authenticating at https://www.orfeus-eu.org/fdsnws/dataselect/1/auth authentication at https://erde.geophysik.uni-muenchen.de/fdsnws/dataselect/1/auth successful getting data from http://erde.geophysik.uni-muenchen.de/fdsnws/dataselect/1/queryauth authentication at https://ws.resif.fr/fdsnws/dataselect/1/auth successful getting data from http://ws.resif.fr/fdsnws/dataselect/1/queryauth authentication at https://www.orfeus-eu.org/fdsnws/dataselect/1/auth successful getting data from http://www.orfeus-eu.org/fdsnws/dataselect/1/queryauth authenticating at https://webservices.ingv.it/fdsnws/dataselect/1/auth got 168448 bytes (application/vnd.fdsn.mseed) from http://eida.ethz.ch/fdsnws/dataselect/1/queryauth authenticating at https://geofon.gfz-potsdam.de/fdsnws/dataselect/1/auth authentication at https://geofon.gfz-potsdam.de/fdsnws/dataselect/1/auth successful getting data from http://geofon.gfz-potsdam.de/fdsnws/dataselect/1/queryauth getting data from http://geofon.gfz-potsdam.de/fdsnws/dataselect/1/queryauth failed with HTTP status code 403: Error 403: Forbidden

 access denied

 Usage details are available from /fdsnws/dataselect/1/

 Request:
 /fdsnws/dataselect/1/queryauth

 Request Submitted:
 2019-11-13T21:27:34.328535

 Service Version:
 1.2.0
 
 authentication at https://webservices.ingv.it/fdsnws/dataselect/1/auth successful
 getting data from http://webservices.ingv.it/fdsnws/dataselect/1/queryauth
 record 72: blockette 1000 not found, stop reading
 got 290816 bytes (application/vnd.fdsn.mseed) from http://ws.resif.fr/fdsnws/dataselect/1/queryauth
 got 229376 bytes (application/vnd.fdsn.mseed) from http://erde.geophysik.uni-muenchen.de/fdsnws/dataselect/1/queryauth
 received no data from http://webservices.ingv.it/fdsnws/dataselect/1/queryauth
 got 173056 bytes (application/vnd.fdsn.mseed) from http://www.orfeus-eu.org/fdsnws/dataselect/1/queryauth
 did not receive data from C4.CERNS.*.HHZ, CH.BALST.*.HHZ, CH.GRYON.*.HHZ, CH.SALAN.*.HHZ, CH.SLE.*.HHZ, CH.TRULL.*.HHZ, Z3.A017A.*.HHZ, Z3.A019A.*.HHZ, Z3.A088B.*.HHZ, Z3.A126A.*.HHZ, Z3.A175A.*.HHZ, Z3.A178A.*.HHZ, Z3.A185A.*.HHZ, Z3.A216A.*.HHZ, Z3.A217A.*.HHZ, Z3.A252A.*.HHZ, Z3.A264A.*.HHZ, Z3.A288A.*.HHZ, Z3.A300A.*.HHZ, Z3.A301A.*.HHZ, Z3.A302A.*.HHZ, Z3.A303A.*.HHZ, Z3.A304A.*.HHZ, Z3.A305A.*.HHZ, Z3.A306A.*.HHZ, Z3.A307A.*.HHZ, Z3.A308A.*.HHZ, Z3.A309A.*.HHZ, Z3.A312A.*.HHZ, Z3.A313A.*.HHZ, Z3.A316A.*.HHZ, Z3.A317A.*.HHZ, Z3.A318A.*.HHZ, Z3.A319A.*.HHZ, Z3.A351A.*.HHZ, Z3.A358A.*.HHZ, Z3.A359A.*.HHZ, Z3.A368A.*.HHZ, Z3.A401A.*.HHZ, Z3.A410A.*.HHZ, Z3.A413A.*.HHZ, Z3.A416A.*.HHZ, Z3.A419A.*.HHZ, Z3.A422A.*.HHZ, Z3.A425A.*.HHZ, Z3.A429A.*.HHZ, ZS.D015.*.HHZ, ZS.D034.*.HHZ, ZS.D035.*.HHZ, ZS.D053.*.HHZ, ZS.D054.*.HHZ, ZS.D055.*.HHZ, ZS.D056.*.HHZ, ZS.D059.*.HHZ, ZS.D060.*.HHZ, ZS.D071.*.HHZ, ZS.D072.*.HHZ, ZS.D073.*.HHZ, ZS.D074.*.HHZ, ZS.D075.*.HHZ, ZS.D076.*.HHZ, ZS.D078.*.HHZ, ZS.D079.*.HHZ, ZS.D092.*.HHZ, ZS.D093.*.HHZ, ZS.D094.*.HHZ, ZS.D095.*.HHZ, ZS.D096.*.HHZ, ZS.D097.*.HHZ, ZS.D098.*.HHZ, ZS.D099.*.HHZ, ZS.D112.*.HHZ, ZS.D113.*.HHZ, ZS.D114.*.HHZ, ZS.D115.*.HHZ, ZS.D116.*.HHZ, ZS.D117.*.HHZ, ZS.D118.*.HHZ, ZS.D119.*.HHZ, ZS.D120.*.HHZ, ZS.D133.*.HHZ, ZS.D134.*.HHZ, ZS.D135.*.HHZ, ZS.D136.*.HHZ, ZS.D137.*.HHZ, ZS.D139.*.HHZ
 retrieving network citation info
 getting routes from http://geofon.gfz-potsdam.de/eidaws/routing/1/query
 getting data from http://ws.resif.fr/fdsnws/station/1/query
 getting data from http://erde.geophysik.uni-muenchen.de/fdsnws/station/1/query
 getting data from http://www.orfeus-eu.org/fdsnws/station/1/query
 getting data from http://webservices.ingv.it/fdsnws/station/1/query
 getting data from http://eida.ethz.ch/fdsnws/station/1/query
 got 267 bytes (text/plain) from http://eida.ethz.ch/fdsnws/station/1/query
 got 144 bytes (text/plain) from http://webservices.ingv.it/fdsnws/station/1/query
 got 152 bytes (text/plain) from http://erde.geophysik.uni-muenchen.de/fdsnws/station/1/query
 got 136 bytes (text/plain) from http://ws.resif.fr/fdsnws/station/1/query
 got 151 bytes (text/plain) from http://www.orfeus-eu.org/fdsnws/station/1/query
 
 You received seismic waveform data from the following network(s):
 C4 CERN Seismic Network
 CH National Seismic Networks of Switzerland
 Z3_2015 AlpArray Seismic Network (AASN) temporary component
 
 Acknowledgment is extremely important for network operators
 providing open data. When preparing publications, please
 cite the data appropriately. The FDSN service at

     http://www.fdsn.org/networks/citation/?networks=C4+CH+Z3_2015

 provides a helpful guide based on available network
 Digital Object Identifiers.
 
 In case of problems with your request, plese use the contact form at

     http://www.orfeus-eu.org/organization/contact/form/?recipient=EIDA
manconia commented 4 years ago

it looks like this is more problematic than I though. Is there a way to make a query with the service dataselect over a defined region (lat,lon,radius)? This works for -y station

fdsnws_fetch -a eidatoken -C HHZ -s 2017-08-23T07:29:00 -e 2017-08-23T07:39:00 -y station -q format=text -q level=channel -q latitude=46.264 -q longitude=9.571 -q maxradius=5 -v -o station.txt

With this I get also all Z3 station as I have the eidatoken

javiquinte commented 4 years ago

Hi @manconia I'm sorry for the delay... (busy days close to the end of the year). I think I've found the cause of the problem.

Very short solution: Use the "-Z" switch with fdsnws_fetch.

Long explanation about why this is happening: _ALPARRAY is a virtual network and when fdsnws_fetch gets the data centres and parameters to request the data does an extra check, what it's usually very good but in this case not. Namely, checks that in the routes received from the Routing Service is present all the information passed as a parameter, what it is a sign that nothing has been lost in the process. However, _ALPARRAY cannot be present in the NSLC codes received because it does not exist like a real network code.

Let's see an example. You request _ALPARRAY and something else to the Routing Service...

http://geofon.gfz-potsdam.de/eidaws/routing/1/query?endtime=2017-08-24&network=_ALPARRAY&starttime=2017-08-23&channel=HHZ&format=post

and you receive

http://eida.ethz.ch/fdsnws/dataselect/1/query CH BERGE HHZ 2017-08-23T00:00:00 2017-08-24T00:00:00 CH EMMET HHZ 2017-08-23T00:00:00 2017-08-24T00:00:00 CH HASLI HHZ 2017-08-23T00:00:00 2017-08-24T00:00:00 CH SIMPL HHZ 2017-08-23T00:00:00 2017-08-24T00:00:00 CH VANNI * HHZ 2017-08-23T00:00:00 2017-08-24T00:00:00

http://www.orfeus-eu.org/fdsnws/dataselect/1/query Z3 A084A HHZ 2017-08-23T00:00:00 2017-08-24T00:00:00 Z3 A009A HHZ 2017-08-23T00:00:00 2017-08-24T00:00:00 Z3 A268A HHZ 2017-08-23T00:00:00 2017-08-24T00:00:00 Z3 A019A HHZ 2017-08-23T00:00:00 2017-08-24T00:00:00

etc. There is no network code _ALPARRAY (only Z3, CH, C4, etc) and that confuses fdsnws_fetch when trying to check that all information is complete and coherent.

manconia commented 4 years ago

Dear Javier,

thanks for the explanation.

Best regards, Andrea

On Thu, Dec 12, 2019 at 11:34 AM Javier Quinteros notifications@github.com wrote:

Hi @manconia https://github.com/manconia I'm sorry for the delay... (busy days close to the end of the year). I think I've found the cause of the problem.

Very short solution: Use the "-Z" switch with fdsnws_fetch.

Long explanation about why this is happening: _ALPARRAY is a virtual network and when fdsnws_fetch gets the data centres and parameters to request the data does an extra check, what it's usually very good but in this case not. Namely, checks that in the routes received from the Routing Service is present all the information passed as a parameter, what it is a sign that nothing has been lost in the process. However, _ALPARRAY cannot be present in the NSLC codes received because it does not exist like a real network code.

Let's see an example. You request _ALPARRAY and something else to the Routing Service...

http://geofon.gfz-potsdam.de/eidaws/routing/1/query?endtime=2017-08-24&network=_ALPARRAY&starttime=2017-08-23&channel=HHZ&format=post

and you receive

http://eida.ethz.ch/fdsnws/dataselect/1/query CH BERGE HHZ 2017-08-23T00:00:00 2017-08-24T00:00:00 CH EMMET HHZ 2017-08-23T00:00:00 2017-08-24T00:00:00 CH HASLI HHZ 2017-08-23T00:00:00 2017-08-24T00:00:00 CH SIMPL HHZ 2017-08-23T00:00:00 2017-08-24T00:00:00 CH VANNI * HHZ 2017-08-23T00:00:00 2017-08-24T00:00:00

http://www.orfeus-eu.org/fdsnws/dataselect/1/query Z3 A084A HHZ 2017-08-23T00:00:00 2017-08-24T00:00:00 Z3 A009A HHZ 2017-08-23T00:00:00 2017-08-24T00:00:00 Z3 A268A HHZ 2017-08-23T00:00:00 2017-08-24T00:00:00 Z3 A019A HHZ 2017-08-23T00:00:00 2017-08-24T00:00:00

etc. There is no network code _ALPARRAY (only Z3, CH, C4, etc) and that confuses fdsnws_fetch when trying to check that all information is complete and coherent.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/EIDA/userfeedback/issues/36?email_source=notifications&email_token=ANYEAXH2EON6E5JEQHJAY7DQYIHUVA5CNFSM4JMVHJ2KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGWHC3A#issuecomment-564949356, or unsubscribe https://github.com/notifications/unsubscribe-auth/ANYEAXHR4QV6ENMFSOHMLB3QYIHUVANCNFSM4JMVHJ2A .

-- Andrea Manconi, PhD

ETH Zürich Dept. of Earth Sciences Engineering Geology NO G3 Sonneggstrasse 5 CH-8092 Zürich, Switzerland

phone: +41 44 633 8405 email: andrea.manconi@erdw.ethz.ch web: http://www.engineeringgeology.ethz.ch/