search

iris-edu / yasmine-stationxml-editor

GNU General Public License v3.0
13 stars 4 forks source link

Support for NRL v2 #42

Open rcasey-earthscope opened 9 months ago

rcasey-earthscope commented 9 months ago

Yasmine was adapted to use NRL v1, which involved a .zip file download of an indexed tree of nominal response RESP files, organized by type, manufacturer, model, and configuration settings.

Yasmine must be adapted to NRL v2, which is a suite of web services and a web tool that provide a dynamic and growing set of those same responses, now combinable on the fly, and delivered in RESP, StationXML, and StationXML-RESP formats.

NRL v2 can return a proper combination of sensor and datalogger response stacking via the web service targeting a specific instance of instrument configuration.

NRL v2 can also return original standalone responses (components) for sensor and datalogger, non combined, with the idea that these could be combined by the client from local store.

The caution for NRL v2 responses is that a sensor component will only contain a rudimentary representation of Stage 1. Datalogger components, which represent subsequent stages in a combined response, also starts its count as Stage 1, with N stages being possible from there. This is a stack shift of 1 that can be misleading, but is at the core of how the system allows combining the two components.

So, the final combination that comes from the web service, with (S)ensor and (D)atalogger components could be represented as:

Component: (S)1 + D(1) + D(2) + D(3).....D(N) Stage Num: 1 2 3 4 0

The final stage number of a complete response in RESP terms is always 0. In StationXML, Stage 0 is referred to as the InstrumentSensitivity.

Gillian currently has a branch (nrl_v2) that can interact with NRL v2 via the Yasmine tool, but there are some integration issues that still need to be worked out. This is probably a good place to start.

A developer working on Instrumentation Services tools is working on a Python library to also permit the interfacing with NRL v2, via ObsPy, to support a number of Python tools. However, the date of completion for this is uncertain, so unsure if this will be available for use soon enough.

https://docs.fdsn.org/projects/stationxml/en/latest/ (StationXML specification) https://ds.iris.edu/NRL/ (NRL v1) - to be retired end of April 2024 https://ds.iris.edu/ds/nrl/ (NRL v2 web page) https://service.iris.edu/irisws/nrl/docs/1/ (NRL v2 web services)

We should make sure that the base URL for NRL v2 can be easily configured by the installer as the NRL URL will change over time.

metempleton commented 9 months ago

I'm testing Gillian's nrl_v2 branch of yasmine and I'd like to test the "integration issues that still need to be worked out" that you mention. Can you include enough details about them for me to test them? I can't tell what they are. Thanks!

rcasey-earthscope commented 9 months ago

Hi Mary- I'm in the middle of some CCP code at the moment, so not prepared to drive Yasmine to get to the spot in question. However, I can best describe it by going through the steps to create a new station and channel, then use the NRL tooling to define your instrumentation.

While the component information is able to be retrieved from NRL v2, I don't think that they combine correctly. Ian has been looking into this as well, but I don't see sign of any code commits to this branch thus far.

Let us know if you see what is going on, because I remember getting some errors popping up in putting together the instrumentation. I think it has to do with the stage mismatch between sensor and datalogger, but I could be wrong.

metempleton commented 9 months ago

Thanks Rob - I've spent some time combining responses and didn't notice anything during fairly extensive response editing. The stationXML validates both with yasmine and with our validator. It's not a drop everything emergency, but I'm interested in how to reproduce what you found when there's time.

is-unavco commented 9 months ago

Thank you Rob and Mary for the additional insights. I am in the process of tracking down and patching these "integration issues", but nothing has popped up in my first pass. Mary, if you do happen to encounter any integration issues in your use of Yasmine please feel free to tag me when opening the ticket.

metempleton commented 9 months ago

Will do Ian - thanks!

Though not a ticket, one thing I've resolved is that the difference between the way v1 and v2 handle stages is addressed properly in the nrl_v2 branch. Maybe this can save you some time. I don't actually know what an "integration issue" is.

More generally, I've been filing issue tickets regarding the main branch for a few weeks now - #35-41 & #43 on. These are also relevant to the nrl_v2 branch.

rcasey-earthscope commented 9 months ago 
An 'integration' issue is where mashing two different softwares together doesn't function correctly or throws errors.   In this case, Yasmine and NRLv2.

I don't know if we need to break out into different tasks, but to start off, let's go with the first use case and see if you reproduce the error I got:

Open up Yasmine.  Create a new XML file and Add a New Network.

A wizard should open up that has you create a Network, enter values in fields, goto Next.
Create a station, enter values in fields, make it just 1 stream, goto Next.
Now we're in the channel section.  Enter values in the fields, goto Next.
Select NRL Library, goto Next.
For the Datalogger selection, I did:  EQMET -> SMA -> 250 Hz
I get a popup:  "An Error Ocurred:  Internal Server Error"

There is an exception stack that appears in the terminal window that I launched from.  I'll share that in a separate comment.

On Mar 1, 2024, at 7:21 AM, Mary Templeton @.***> wrote:

Will do Ian - thanks!

Though not a ticket, one thing I've resolved is that the difference between the way v1 and v2 handle stages is addressed properly in the nrl_v2 branch. Maybe this can save you some time. I don't actually know what an "integration issue" is.

More generally, I've been filing issue tickets regarding the main branch for a few weeks now - #35 https://github.com/iris-edu/yasmine-stationxml-editor/issues/35-41 & #43 https://github.com/iris-edu/yasmine-stationxml-editor/issues/43 on. These are also relevant to the nrl_v2 branch.

— Reply to this email directly, view it on GitHub https://github.com/iris-edu/yasmine-stationxml-editor/issues/42#issuecomment-1973378165, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFL4VN3V4H7TSQ4FEFCPRTTYWCMGBAVCNFSM6AAAAABDG66M7CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNZTGM3TQMJWGU. You are receiving this because you were assigned.

rcasey-earthscope commented 9 months ago 
yasmine-backend   | 2024-03-01 22:02:18,791 - INFO - web - 200 GET /api/nrl/sensors/0?_dc=1709330538680 (172.18.0.2) 76.62ms
yasmine-backend   | INFO:tornado.access:200 GET /api/nrl/sensors/0?_dc=1709330538680 (172.18.0.2) 76.62ms
yasmine-backend   | 2024-03-01 22:02:18,868 - INFO - web - 200 GET /api/nrl/dataloggers/0?_dc=1709330538682 (172.18.0.2) 137.01ms
yasmine-backend   | INFO:tornado.access:200 GET /api/nrl/dataloggers/0?_dc=1709330538682 (172.18.0.2) 137.01ms
yasmine-backend   | 2024-03-01 22:03:31,417 - ERROR - web - Uncaught exception GET /api/nrl/datalogger/response/?_dc=1709330611373&keys=EQMet&keys=SMA&keys=250%20Hz (172.18.0.2)
yasmine-backend   | HTTPServerRequest(protocol='http', host='yasmine-backend:80', method='GET', uri='/api/nrl/datalogger/response/?_dc=1709330611373&keys=EQMet&keys=SMA&keys=250%20Hz', version='HTTP/1.1', remote_ip='172.18.0.2')
yasmine-backend   | Traceback (most recent call last):
yasmine-backend   |   File "/usr/local/lib/python3.12/site-packages/obspy/clients/nrl/client.py", line 68, in __init__
yasmine-backend   |     self.sensors = self._parse_ini(sensor_index)
yasmine-backend   |                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
yasmine-backend   |   File "/usr/local/lib/python3.12/site-packages/obspy/clients/nrl/client.py", line 129, in _parse_ini
yasmine-backend   |     cp = self._get_cp_from_ini(path)
yasmine-backend   |          ^^^^^^^^^^^^^^^^^^^^^^^^^^^
yasmine-backend   |   File "/usr/local/lib/python3.12/site-packages/obspy/clients/nrl/client.py", line 320, in _get_cp_from_ini
yasmine-backend   |     with codecs.open(path, mode='r', encoding='UTF-8') as f:
yasmine-backend   |          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
yasmine-backend   |   File "", line 918, in open
yasmine-backend   | FileNotFoundError: [Errno 2] No such file or directory: '/opt/yasmine/data/_media/nrl/content/NRL/sensors/index.txt'
yasmine-backend   |
yasmine-backend   | During handling of the above exception, another exception occurred:
yasmine-backend   |
yasmine-backend   | Traceback (most recent call last):
yasmine-backend   |   File "/usr/local/lib/python3.12/site-packages/tornado/web.py", line 1790, in _execute
yasmine-backend   |     result = await result
yasmine-backend   |              ^^^^^^^^^^^^
yasmine-backend   |   File "/usr/local/lib/python3.12/site-packages/tornado/gen.py", line 780, in run
yasmine-backend   |     yielded = self.gen.throw(exc)
yasmine-backend   |               ^^^^^^^^^^^^^^^^^^^
yasmine-backend   |   File "/opt/yasmine/src/yasmine/app/handlers/base.py", line 79, in get
yasmine-backend   |     res = yield self.async_call(self.async_get, *args, **kwargs)
yasmine-backend   |           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
yasmine-backend   |   File "/usr/local/lib/python3.12/site-packages/tornado/gen.py", line 767, in run
yasmine-backend   |     value = future.result()
yasmine-backend   |             ^^^^^^^^^^^^^^^
yasmine-backend   |   File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 58, in run
yasmine-backend   |     result = self.fn(*self.args, **self.kwargs)
yasmine-backend   |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
yasmine-backend   |   File "/opt/yasmine/src/yasmine/app/handlers/base.py", line 63, in async_call
yasmine-backend   |     return func(*args, **kwargs)
yasmine-backend   |            ^^^^^^^^^^^^^^^^^^^^^
yasmine-backend   |   File "/opt/yasmine/src/yasmine/app/handlers/xml_nrl.py", line 57, in async_get
yasmine-backend   |     return self.helper.get_datalogger_response_str(self.get_arguments('keys'))
yasmine-backend   |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
yasmine-backend   |   File "/opt/yasmine/src/yasmine/app/helpers/nrl/nrl_helper.py", line 71, in get_datalogger_response_str
yasmine-backend   |     path = self._build_path(self.nrl.dataloggers, datalogger_keys)
yasmine-backend   |                             ^^^^^^^^
yasmine-backend   |   File "/opt/yasmine/src/yasmine/app/helpers/nrl/nrl_helper.py", line 118, in nrl
yasmine-backend   |     self._nrl = NRL(os.path.join(self.content_folder,'NRL'))
yasmine-backend   |                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
yasmine-backend   |   File "/usr/local/lib/python3.12/site-packages/obspy/clients/nrl/client.py", line 312, in __init__
yasmine-backend   |     super(self.__class__, self).__init__()
yasmine-backend   |   File "/usr/local/lib/python3.12/site-packages/obspy/clients/nrl/client.py", line 76, in __init__
yasmine-backend   |     self.sensors = self._parse_ini(sensor_index)
yasmine-backend   |                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
yasmine-backend   |   File "/usr/local/lib/python3.12/site-packages/obspy/clients/nrl/client.py", line 129, in _parse_ini
yasmine-backend   |     cp = self._get_cp_from_ini(path)
yasmine-backend   |          ^^^^^^^^^^^^^^^^^^^^^^^^^^^
yasmine-backend   |   File "/usr/local/lib/python3.12/site-packages/obspy/clients/nrl/client.py", line 320, in _get_cp_from_ini
yasmine-backend   |     with codecs.open(path, mode='r', encoding='UTF-8') as f:
yasmine-backend   |          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
yasmine-backend   |   File "", line 918, in open
yasmine-backend   | FileNotFoundError: [Errno 2] No such file or directory: '/opt/yasmine/data/_media/nrl/content/NRL/sensor/index.txt'
yasmine-backend   | ERROR:tornado.application:Uncaught exception GET /api/nrl/datalogger/response/?_dc=1709330611373&keys=EQMet&keys=SMA&keys=250%20Hz (172.18.0.2)
yasmine-backend   | HTTPServerRequest(protocol='http', host='yasmine-backend:80', method='GET', uri='/api/nrl/datalogger/response/?_dc=1709330611373&keys=EQMet&keys=SMA&keys=250%20Hz', version='HTTP/1.1', remote_ip='172.18.0.2')
yasmine-backend   | Traceback (most recent call last):
yasmine-backend   |   File "/usr/local/lib/python3.12/site-packages/obspy/clients/nrl/client.py", line 68, in __init__
yasmine-backend   |     self.sensors = self._parse_ini(sensor_index)
yasmine-backend   |                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
yasmine-backend   |   File "/usr/local/lib/python3.12/site-packages/obspy/clients/nrl/client.py", line 129, in _parse_ini
yasmine-backend   |     cp = self._get_cp_from_ini(path)
yasmine-backend   |          ^^^^^^^^^^^^^^^^^^^^^^^^^^^
yasmine-backend   |   File "/usr/local/lib/python3.12/site-packages/obspy/clients/nrl/client.py", line 320, in _get_cp_from_ini
yasmine-backend   |     with codecs.open(path, mode='r', encoding='UTF-8') as f:
yasmine-backend   |          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
yasmine-backend   |   File "", line 918, in open
yasmine-backend   | FileNotFoundError: [Errno 2] No such file or directory: '/opt/yasmine/data/_media/nrl/content/NRL/sensors/index.txt'
yasmine-backend   |
yasmine-backend   | During handling of the above exception, another exception occurred:
yasmine-backend   |
yasmine-backend   | Traceback (most recent call last):
yasmine-backend   |   File "/usr/local/lib/python3.12/site-packages/tornado/web.py", line 1790, in _execute
yasmine-backend   |     result = await result
yasmine-backend   |              ^^^^^^^^^^^^
yasmine-backend   |   File "/usr/local/lib/python3.12/site-packages/tornado/gen.py", line 780, in run
yasmine-backend   |     yielded = self.gen.throw(exc)
yasmine-backend   |               ^^^^^^^^^^^^^^^^^^^
yasmine-backend   |   File "/opt/yasmine/src/yasmine/app/handlers/base.py", line 79, in get
yasmine-backend   |     res = yield self.async_call(self.async_get, *args, **kwargs)
yasmine-backend   |           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
yasmine-backend   |   File "/usr/local/lib/python3.12/site-packages/tornado/gen.py", line 767, in run
yasmine-backend   |     value = future.result()
yasmine-backend   |             ^^^^^^^^^^^^^^^
yasmine-backend   |   File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 58, in run
yasmine-backend   |     result = self.fn(*self.args, **self.kwargs)
yasmine-backend   |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
yasmine-backend   |   File "/opt/yasmine/src/yasmine/app/handlers/base.py", line 63, in async_call
yasmine-backend   |     return func(*args, **kwargs)
yasmine-backend   |            ^^^^^^^^^^^^^^^^^^^^^
yasmine-backend   |   File "/opt/yasmine/src/yasmine/app/handlers/xml_nrl.py", line 57, in async_get
yasmine-backend   |     return self.helper.get_datalogger_response_str(self.get_arguments('keys'))
yasmine-backend   |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
yasmine-backend   |   File "/opt/yasmine/src/yasmine/app/helpers/nrl/nrl_helper.py", line 71, in get_datalogger_response_str
yasmine-backend   |     path = self._build_path(self.nrl.dataloggers, datalogger_keys)
yasmine-backend   |                             ^^^^^^^^
yasmine-backend   |   File "/opt/yasmine/src/yasmine/app/helpers/nrl/nrl_helper.py", line 118, in nrl
yasmine-backend   |     self._nrl = NRL(os.path.join(self.content_folder,'NRL'))
yasmine-backend   |                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
yasmine-backend   |   File "/usr/local/lib/python3.12/site-packages/obspy/clients/nrl/client.py", line 312, in __init__
yasmine-backend   |     super(self.__class__, self).__init__()
yasmine-backend   |   File "/usr/local/lib/python3.12/site-packages/obspy/clients/nrl/client.py", line 76, in __init__
yasmine-backend   |     self.sensors = self._parse_ini(sensor_index)
yasmine-backend   |                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
yasmine-backend   |   File "/usr/local/lib/python3.12/site-packages/obspy/clients/nrl/client.py", line 129, in _parse_ini
yasmine-backend   |     cp = self._get_cp_from_ini(path)
yasmine-backend   |          ^^^^^^^^^^^^^^^^^^^^^^^^^^^
yasmine-backend   |   File "/usr/local/lib/python3.12/site-packages/obspy/clients/nrl/client.py", line 320, in _get_cp_from_ini
yasmine-backend   |     with codecs.open(path, mode='r', encoding='UTF-8') as f:
yasmine-backend   |          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
yasmine-backend   |   File "", line 918, in open
yasmine-backend   | FileNotFoundError: [Errno 2] No such file or directory: '/opt/yasmine/data/_media/nrl/content/NRL/sensor/index.txt'
yasmine-backend   | 2024-03-01 22:03:31,446 - ERROR - web - 500 GET /api/nrl/datalogger/response/?_dc=1709330611373&keys=EQMet&keys=SMA&keys=250%20Hz (172.18.0.2) 38.35ms
yasmine-backend   | ERROR:tornado.access:500 GET /api/nrl/datalogger/response/?_dc=1709330611373&keys=EQMet&keys=SMA&keys=250%20Hz (172.18.0.2) 38.35ms
rcasey-earthscope commented 9 months ago

Of note is that the sensors.json and dataloggers.json are found in /Users/rob/Desktop/yasmine/install_run_Jan_2024/yasmine-stationxml-editor-nrl_v2/data/_media/nrl. I am able to make selections from a list, so the navigation is not a problem. However, the call to /api/nrl/datalogger/response seems to not yield anything.

When looking at the nrlv2 diff, I note that it looks like the entire .zip file of NRLv2 is downloaded from the web service. I guess this is fine for a first go, but I thought that Yasmine was going to call to the web service itself. Is there some kind of initialization I need to do?

metempleton commented 9 months ago

Thanks Rob!

My installation must be different, as I succeeded getting the EQMet SMA 250 Hz response with this console message:

yasmine-backend | 2024-03-01 22:47:55,154 - INFO - web - 200 GET /api/nrl/datalogger/response/?_dc=1709333274755&keys=EQMet&keys=SMA&keys=250%20Hz (172.18.0.2) 388.14ms yasmine-backend | INFO:tornado.access:200 GET /api/nrl/datalogger/response/?_dc=1709333274755&keys=EQMet&keys=SMA&keys=250%20Hz (172.18.0.2) 388.14ms

FWIW, Gillian's branch pulls the full cached .zip file from the NRL ws and this takes time (X min?) to download and unzip. But it only does this once per day; I'm not sure what time. If yasmine is pulling a fresh copy, you'll need to wait until the following "docker-compose up" console message reads:

yasmine-backend | INFO:yasmine.app.helpers.nrl.nrl_helper:Creating an NRL key files

before proceeding, and there is nothing to tell you that this needs to be done currently.

You can also check whether the NRL has completed downloading by listing yasmine-stationxml-editor/data/_media/nrl/content/NRL - the datalogger and sensor dirs.

gillian-earthscope commented 9 months ago

The full console messages are:

yasmine-backend | INFO:yasmine.app.helpers.nrl.nrl_helper:Unzipping NRL

yasmine-backend | INFO:yasmine.app.helpers.nrl.nrl_helper:NRL has been unzipped

yasmine-backend | INFO:yasmine.app.helpers.nrl.nrl_helper:Creating an NRL key files

yasmine-backend | INFO:yasmine.app.helpers.nrl.nrl_helper:NRL key files have been created

It should start the download when you start running the docker container.

Yes, we will want Yasmine to use the web service when possible but replacing the NRL version 1 zip file download with version 2 was the easiest way to get started working with version 2. As far as I know, Omid is still working on a Python library for using the NRL web service.

cheers,

Gillian

On Fri, Mar 1, 2024 at 2:54 PM Mary Templeton @.***> wrote:

Thanks Rob!

My installation must be different, as I succeeded getting the EQMet SMA 250 Hz response with this console message:

yasmine-backend | 2024-03-01 22:47:55,154 - INFO - web - 200 GET /api/nrl/datalogger/response/?_dc=1709333274755&keys=EQMet&keys=SMA&keys=250%20Hz (172.18.0.2) 388.14ms yasmine-backend | INFO:tornado.access:200 GET /api/nrl/datalogger/response/?_dc=1709333274755&keys=EQMet&keys=SMA&keys=250%20Hz (172.18.0.2) 388.14ms

FWIW, Gillian's branch pulls the full cached .zip file from the NRL ws and this takes time (X min?) to download and unzip. But it only does this once per day; I'm not sure what time. If yasmine is pulling a fresh copy, you'll need to wait until the following "docker-compose up" console message reads:

yasmine-backend | INFO:yasmine.app.helpers.nrl.nrl_helper:Creating an NRL key files

before proceeding, and there is nothing to tell you that this needs to be done currently.

You can also check whether the NRL has completed downloading by listing yasmine-stationxml-editor/data/_media/nrl/content/NRL - the datalogger and sensor dirs.

— Reply to this email directly, view it on GitHub https://github.com/iris-edu/yasmine-stationxml-editor/issues/42#issuecomment-1974040796, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFNYF2QIYZMF6RG6EJQXC7LYWEBKDAVCNFSM6AAAAABDG66M7CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNZUGA2DANZZGY . You are receiving this because you are subscribed to this thread.Message ID: @.***>

gillian-earthscope commented 9 months ago

One issue is that if you had the NRL previously downloaded, you can work with Yasmine immediately but when it starts unpacking the new download while you are working then you'll start getting errors. The other of course if you don't have a copy initially, you get errors right away and it's not obvious that you have to wait 10+ minutes before it will be useable.

-Gillian

On Fri, Mar 1, 2024 at 4:42 PM Gillian Sharer @.***> wrote:

The full console messages are:

yasmine-backend | INFO:yasmine.app.helpers.nrl.nrl_helper:Unzipping NRL

yasmine-backend | INFO:yasmine.app.helpers.nrl.nrl_helper:NRL has been unzipped

yasmine-backend | INFO:yasmine.app.helpers.nrl.nrl_helper:Creating an NRL key files

yasmine-backend | INFO:yasmine.app.helpers.nrl.nrl_helper:NRL key files have been created

It should start the download when you start running the docker container.

Yes, we will want Yasmine to use the web service when possible but replacing the NRL version 1 zip file download with version 2 was the easiest way to get started working with version 2. As far as I know, Omid is still working on a Python library for using the NRL web service.

cheers,

Gillian

On Fri, Mar 1, 2024 at 2:54 PM Mary Templeton @.***> wrote:

Thanks Rob!

My installation must be different, as I succeeded getting the EQMet SMA 250 Hz response with this console message:

yasmine-backend | 2024-03-01 22:47:55,154 - INFO - web - 200 GET /api/nrl/datalogger/response/?_dc=1709333274755&keys=EQMet&keys=SMA&keys=250%20Hz (172.18.0.2) 388.14ms yasmine-backend | INFO:tornado.access:200 GET /api/nrl/datalogger/response/?_dc=1709333274755&keys=EQMet&keys=SMA&keys=250%20Hz (172.18.0.2) 388.14ms

FWIW, Gillian's branch pulls the full cached .zip file from the NRL ws and this takes time (X min?) to download and unzip. But it only does this once per day; I'm not sure what time. If yasmine is pulling a fresh copy, you'll need to wait until the following "docker-compose up" console message reads:

yasmine-backend | INFO:yasmine.app.helpers.nrl.nrl_helper:Creating an NRL key files

before proceeding, and there is nothing to tell you that this needs to be done currently.

You can also check whether the NRL has completed downloading by listing yasmine-stationxml-editor/data/_media/nrl/content/NRL - the datalogger and sensor dirs.

— Reply to this email directly, view it on GitHub https://github.com/iris-edu/yasmine-stationxml-editor/issues/42#issuecomment-1974040796, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFNYF2QIYZMF6RG6EJQXC7LYWEBKDAVCNFSM6AAAAABDG66M7CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNZUGA2DANZZGY . You are receiving this because you are subscribed to this thread.Message ID: @.***>

rcasey-earthscope commented 9 months ago 
Thanks for the feedback everyone.  I'll see if I can get the NRL content to update and see if that addresses the issue.

All of these behaviors are something we should document or improve.  I'll add a new issue to the project space regarding the fact that I can't tell when Yasmine is actually up and ready, since the logging to the screen is so cryptic.

I miss the good old days when we were able to launch code from an icon (cross-platform).

-Rob

On Mar 1, 2024, at 5:00 PM, Gillian Sharer @.***> wrote:

One issue is that if you had the NRL previously downloaded, you can work with Yasmine immediately but when it starts unpacking the new download while you are working then you'll start getting errors. The other of course if you don't have a copy initially, you get errors right away and it's not obvious that you have to wait 10+ minutes before it will be useable.

-Gillian

On Fri, Mar 1, 2024 at 4:42 PM Gillian Sharer @.***> wrote:

The full console messages are:

yasmine-backend | INFO:yasmine.app.helpers.nrl.nrl_helper:Unzipping NRL

yasmine-backend | INFO:yasmine.app.helpers.nrl.nrl_helper:NRL has been unzipped

yasmine-backend | INFO:yasmine.app.helpers.nrl.nrl_helper:Creating an NRL key files

yasmine-backend | INFO:yasmine.app.helpers.nrl.nrl_helper:NRL key files have been created

It should start the download when you start running the docker container.

Yes, we will want Yasmine to use the web service when possible but replacing the NRL version 1 zip file download with version 2 was the easiest way to get started working with version 2. As far as I know, Omid is still working on a Python library for using the NRL web service.

cheers,

Gillian

On Fri, Mar 1, 2024 at 2:54 PM Mary Templeton @.***> wrote:

Thanks Rob!

My installation must be different, as I succeeded getting the EQMet SMA 250 Hz response with this console message:

yasmine-backend | 2024-03-01 22:47:55,154 - INFO - web - 200 GET /api/nrl/datalogger/response/?_dc=1709333274755&keys=EQMet&keys=SMA&keys=250%20Hz (172.18.0.2) 388.14ms yasmine-backend | INFO:tornado.access:200 GET /api/nrl/datalogger/response/?_dc=1709333274755&keys=EQMet&keys=SMA&keys=250%20Hz (172.18.0.2) 388.14ms

FWIW, Gillian's branch pulls the full cached .zip file from the NRL ws and this takes time (X min?) to download and unzip. But it only does this once per day; I'm not sure what time. If yasmine is pulling a fresh copy, you'll need to wait until the following "docker-compose up" console message reads:

yasmine-backend | INFO:yasmine.app.helpers.nrl.nrl_helper:Creating an NRL key files

before proceeding, and there is nothing to tell you that this needs to be done currently.

You can also check whether the NRL has completed downloading by listing yasmine-stationxml-editor/data/_media/nrl/content/NRL - the datalogger and sensor dirs.

— Reply to this email directly, view it on GitHub https://github.com/iris-edu/yasmine-stationxml-editor/issues/42#issuecomment-1974040796, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFNYF2QIYZMF6RG6EJQXC7LYWEBKDAVCNFSM6AAAAABDG66M7CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNZUGA2DANZZGY . You are receiving this because you are subscribed to this thread.Message ID: @.***>

— Reply to this email directly, view it on GitHub https://github.com/iris-edu/yasmine-stationxml-editor/issues/42#issuecomment-1974149936, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFL4VN4XC6UHDRRIBBXPNXDYWEQBVAVCNFSM6AAAAABDG66M7CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNZUGE2DSOJTGY. You are receiving this because you were assigned.

rcasey-earthscope commented 9 months ago 
I was able to get through the NRL steps with only a minor hitch.  Apparently I had a warning in the text window (and not in the UI).

 WARNING (norm_resp): computed and reported sensitivities differ by more than 5 percent.

I now somehow got myself stuck on the last page where I can save the results to the network user library, the station user library, and the channel user library.  When I hit Complete Wizard, it pops up an error and asks me to select a user library.  I can select all checkboxes, I can select one, doesn't matter.  If I roll back in the wizard and move forward again, same thing.  I've now got myself trapped.  If I reload the page, it will remove everything I did and I have to start over again.   Looks like I have more issues to file.

However, I think that the NRL integration issue is dealt with.  If Mary's tests show that its valid response, then we may not have more of this NRL issue to address.  We're done with it, right?

-Rob

On Mar 1, 2024, at 5:00 PM, Gillian Sharer @.***> wrote:

One issue is that if you had the NRL previously downloaded, you can work with Yasmine immediately but when it starts unpacking the new download while you are working then you'll start getting errors. The other of course if you don't have a copy initially, you get errors right away and it's not obvious that you have to wait 10+ minutes before it will be useable.

-Gillian

On Fri, Mar 1, 2024 at 4:42 PM Gillian Sharer @.***> wrote:

The full console messages are:

yasmine-backend | INFO:yasmine.app.helpers.nrl.nrl_helper:Unzipping NRL

yasmine-backend | INFO:yasmine.app.helpers.nrl.nrl_helper:NRL has been unzipped

yasmine-backend | INFO:yasmine.app.helpers.nrl.nrl_helper:Creating an NRL key files

yasmine-backend | INFO:yasmine.app.helpers.nrl.nrl_helper:NRL key files have been created

It should start the download when you start running the docker container.

Yes, we will want Yasmine to use the web service when possible but replacing the NRL version 1 zip file download with version 2 was the easiest way to get started working with version 2. As far as I know, Omid is still working on a Python library for using the NRL web service.

cheers,

Gillian

On Fri, Mar 1, 2024 at 2:54 PM Mary Templeton @.***> wrote:

Thanks Rob!

My installation must be different, as I succeeded getting the EQMet SMA 250 Hz response with this console message:

yasmine-backend | 2024-03-01 22:47:55,154 - INFO - web - 200 GET /api/nrl/datalogger/response/?_dc=1709333274755&keys=EQMet&keys=SMA&keys=250%20Hz (172.18.0.2) 388.14ms yasmine-backend | INFO:tornado.access:200 GET /api/nrl/datalogger/response/?_dc=1709333274755&keys=EQMet&keys=SMA&keys=250%20Hz (172.18.0.2) 388.14ms

FWIW, Gillian's branch pulls the full cached .zip file from the NRL ws and this takes time (X min?) to download and unzip. But it only does this once per day; I'm not sure what time. If yasmine is pulling a fresh copy, you'll need to wait until the following "docker-compose up" console message reads:

yasmine-backend | INFO:yasmine.app.helpers.nrl.nrl_helper:Creating an NRL key files

before proceeding, and there is nothing to tell you that this needs to be done currently.

You can also check whether the NRL has completed downloading by listing yasmine-stationxml-editor/data/_media/nrl/content/NRL - the datalogger and sensor dirs.

— Reply to this email directly, view it on GitHub https://github.com/iris-edu/yasmine-stationxml-editor/issues/42#issuecomment-1974040796, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFNYF2QIYZMF6RG6EJQXC7LYWEBKDAVCNFSM6AAAAABDG66M7CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNZUGA2DANZZGY . You are receiving this because you are subscribed to this thread.Message ID: @.***>

— Reply to this email directly, view it on GitHub https://github.com/iris-edu/yasmine-stationxml-editor/issues/42#issuecomment-1974149936, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFL4VN4XC6UHDRRIBBXPNXDYWEQBVAVCNFSM6AAAAABDG66M7CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNZUGE2DSOJTGY. You are receiving this because you were assigned.

rcasey-earthscope commented 8 months ago

Since the NRL v2 download of the zip file is working, we want to review that NRL navigation is functioning correctly and that responses are being generated properly.