FDSN-NRT - realtime data query drops (from september 2021)

[salichon]

Email from Mike Gabbitas.mikenz4591@gmail.com

Context:

• FDSN-nrt used along with Swarm software by a remote user

Issue:

• FDSN-nrtquery drops for many stations for a small time often since late september 2021 and still goes on
• Drops look like failing at a round hour
• The drops are occurring on querying for actual real-time data (short delay with the measure on the field)
  "The most interesting thing I noticed is these only started becoming a regular occurrence from Friday 24th at 1 AM NZST, and having run this Swarm layout as it is currently for over 2 years I don't think the number of stations makes any difference Mike"

Example:

• Evidences of these drops are shown here : https://www.youtube.com/watch?v=8goP58spRvA

Is the fdsn-nrt service designed to perform such real time queries (and to work with swram or other process) ? Any Comments suggestion @sue-h-gns @junghao about this issue?

thanks cheers jerome

FYI @JonoHanson @elidana @rumachan

---

--- emails follow ----

Thanks a lot  Mike

the most important thing is :
"The drops continue at the present time."
 I ll look into this service more in details

Would you try with fewer stations under swarm?
in order to identify if it is not related to the number of queries performed with the service.

thanks Cheers!

jerome

---

From: Michael Gabbitas <mikenz4591@gmail.com>
Sent: 04 October 2021 17:08
To: Jerome Salichon <J.Salichon@gns.cri.nz>
Subject: Re: Data Drops Geonet Servers (FDNS-NRT)

 

  | Note: External sender: -- | --

Hi again,
I had a look through my recordings which only go back 8 days, it appears these data drops started Friday 24th at 1 AM NZST and were regular throughout the day, seemingly every hour on the hour.
They continued through the weekend but with less frequency until the present.
To explain what I am talking about, these are Spectrorams, I have re recorded some of them to show you how they appear.

I only do this for personal use, but share with others here in NZ through Twitch TV, my broadcasts run 24/7 and have done so since April (ish) 2019 with permission from Geonet.

Had seen a few data drops previously over the 2.5 years but never as much as in the last 6 days.

I also do a Global spectrogram stream with 3000 subscribers worldwide on Twitch TV called DEC, bringing in Spectrogams through Swarm via IRIS and Geofon. It has a NZ block of 5 stations from Raoul Is. down to Te Anau and the data drops are occurring there also, but only Geonet data not the others.
It could well be my ISP that is having problems, I don't know, but I thought I would ask you guys first.
Here is the video I mentioned viewable on You Tube with some examples of the data drops, there were many more over the past 6 days, too many to re-record all of them.
https://youtu.be/8goP58spRvA
The actual spectrogram segments are sped up to 16 times speed so I could get them all in the one video, you can slow the video down using You Tube settings
If you click on the Cat Icon Tom WIlson there on the page it will take you to the channel menu, click on "videos" and you can see some significant earthquakes I had previously recorded
Thanks for your attention
Mike Gabbitas

[junghao]

@salichon

Technical analyse of this issue:

• This issue happens at 00 minute each hour due to a bug in the system
• The gap only observable when you do live streaming crossing 00 minute
• The gap will be backfilled after 5 minute later (the system does it), thus the records in the system is still having 100% completeness

We've fixed the bug so no gaps happen anymore.

[salichon]

• Will communicate to Mike (and close) Thanks cheers jerome