junghao
commented
1 month ago Developer's note:
Test against Fastly and origin:
Fastly:
% curl -o test2.ms "https://service.geonet.org.nz/fdsnws/dataselect/1/query?station=WQAB&channel=HN?&endtime=2024-03-16T02:10:56&starttime=2024-03-16T01:56:56&format=miniseed&nodata=204"
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 10.2M 0 10.2M 0 0 152k 0 --:--:-- 0:01:09 --:--:-- 0
curl: (92) HTTP/2 stream 1 was not closed cleanly: PROTOCOL_ERROR (err 1)Origin:
% curl -o test.ms "http://tf-prod-fdsn-web-2006946694.ap-southeast-2.elb.amazonaws.com/fdsnws/dataselect/1/query?station=WQAB&channel=HN?&endtime=2024-03-16T02:10:56&starttime=2024-03-16T01:56:56&format=miniseed&nodata=204"
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 10.2M 0 10.2M 0 0 104k 0 --:--:-- 0:01:40 --:--:-- 335We can see that download from origin takes longer, and I have observed that the last 30 seconds of download against origin is a bit stalled (average speed kept dropping due to no packet received, current speed became 0). However, the download finished after the 20~30seconds stall.
Then check the downloaded file size:
% ls -l test*.ms
-rw-r--r-- 1 howardw GNS\g_howardw 10785280 31 Jul 10:18 test.ms
-rw-r--r-- 1 howardw GNS\g_howardw 10783950 31 Jul 10:19 test2.msThe file downloaded via Fastly is 1330 bytes short.
I'm guessing this query returned many files, and many of the files don't contain the data during the time period user requested. Thus the last packet took quite long to scan thru many files, which caused a timeout for Fastly.
Next step:
- Check result set that database returns, see if they're reasonable.
- Have a look at those files, see if anything unusual.
bpeng
@ozym reported an issues
Just noticing some errors turning up with the FDSN service, they are appearing in some of the strong motion processing. They look like:
in amongst valid data. This query was enough to trigger one:
But it doesn't seem to happen all the time, this query run again may be okay. The ECS strong build application logs looks like below, the "PROTOCOL_ERROR is coming from the dataservice of FDSN".