jpjones76 / SeisIO.jl

Julia language support for geophysical time series data
http://seisio.readthedocs.org
Other
47 stars 20 forks source link

MSEED from IRIS directly dumped to disk have strange amplitude #45

Closed lermert closed 4 years ago

lermert commented 4 years ago

Hi! I recently noticed that when I ran the following command:

stream = get_data("IRIS", "CI.SVD..BHZ", s=DateTime("2007-01-01"), t=3600, w=true, v=1, rr=false, unscale=false)

The resulting stream has amplitudes in the range of 10^-6 rather than 10^3 or so as I would expect. Using the old fetchdata scripts (https://seiscode.iris.washington.edu/projects/ws-fetch-scripts/wiki) from IRIS returns miniseeds with correct amplitude so I am wondering if this issue can be fixed within SeisIO. Happy to provide more info, I am running SeisIO v1.1.0 on different ubuntu versions. Thanks for any help with the issue.

jpjones76 commented 4 years ago

This could use more documentation on my part, but it's ultimately a server-side issue.

IRISWS timeseries doesn't set the gain constant in data requests. To ensure that the :gain field is accurate in SeisIO, get_data("IRIS", ... ) removes the ("Stage 0") gain, even with "unscale=false". In terms of how SeisIO interacts with IRISWS timeseries, it includes KW scale=AUTO in the URL string.

For example, have a look at the unscaled equivalent:

using HTTP
url = "http://service.iris.edu/irisws/timeseries/1/query?net=CI&sta=SVD&loc=--&cha=BHZ&start=2007-01-01T00:00:00&end=2007-01-01T01:00:00&format=miniseed"
req = HTTP.request("GET", url)
io = open("tmp.mseed", "w")
write(io, req.body)
close(io)
S = read_data("mseed", "tmp.mseed")

It has the expected amplitude range, but S.gain = 1.0. Gain isn't set in the mini-SEED file; if you read it into any other program, it has the same problem. Other file formats also do this. In fact, there's no way to get the gain without calling other IRISWS services (or FDSN, which makes assumptions about channel naming correspondence).

If you absolutely need an unscaled time series with gain constant, we can talk about ways to implement that (I think IRISWS resp should have it). Remember, I'm always willing to change default SeisIO behavior for a demonstrated widespread need, or if the current behavior causes problems with data analysis.

jpjones76 commented 4 years ago

Wait. How can you be running SeisIO v1.1.0?

lermert commented 4 years ago

Hi, ok! I understand that MSEED doesn't include any information on gain. Compared to other downloaders though the behaviour was unexpected and caused some issues in my processing flow that are easily avoided by accounting for the gain removal. So I'd suggest to specify it more explicitly in the documentation. I can add some lines if that's helpful!

jpjones76 commented 4 years ago

The IRIS timeseries web service is what excludes the gain info. It's not an MSEED issue. As my comment above explained, the requested data format bears no relevance.

If you request MSEED data from an FDSN server, i.e., get_data("FDSN", ... ), the default keywords will return correct gain info.

jpjones76 commented 4 years ago

I've updated the documentation on the master branch to reflect this behavior.

The IRISWS timeseries service recently changed how they record gain in data requests: at the moment, the handling depends on the data format (previously, gain was uniformly 1.0). I've tried to explain the current behavior in CHANGELOG.md, and how SeisIO handles gain in a format-agnostic way, but this looks like a bug on their end. I've emailed them for clarification about their intent for gain behavior, and I'll leave this issue open until I get a reply.

lermert commented 4 years ago

Hi Josh! Thanks a lot for updating the documentation and contacting IRIS. Let's hope they'll get back to us about it.