search

asascience / ott-coops

Project Tracking for OTT Co-Ops
1 stars 0 forks source link

Differentiate CO-OPS Stations with Just Tide Predictions vs Ones with Water Level Measurements #434

Open jamesdoyle21 opened 2 years ago

jamesdoyle21 commented 2 years ago

OceansMap Bug Tracker line 40

This is an issue that has layers and is causing problems down the line. The problem is that CO-OPS stations labeled "Water Levels" contain 2 types of station: stations with both measured water level data and tidal predictions and stations with only tidal predictions. We need to differentiate these because the stations with tidal predictions only have no real-time data - just predictions.

Matt, Chris and I did not have a clear solution to this. Ideally, we would add a 5th category to the CO-OPS stations titled "Tide Predictions" which contains all the stations with only tide predictions and nothing else. This is a challenging problem though apparently, so it may require some discussion of how to do this here.

This issue directly relates to #435. If we can create a separate group for the tide prediction only stations, we can remove these from the validation tool completely to solve #435.

image

kvmartin001 commented 2 years ago

For https://github.com/asascience/ott-coops/issues/429 I'm adding the category as an actual data attribute, so the category will be readable by the validation tool.

jamesdoyle21 commented 2 years ago

The red Water Levels stations are coming up orange on my map. This might be because all the statuions are out of date, but either way, this should be easier to read.

Also, I am not seeing this change in the Client Demo instance

Putting this back into In Progress @kvmartin001

image

jamesdoyle21 commented 2 years ago

The other thing is there are many stations with Water Level listed as a parameter, but never have data for this.

@cpaternostro can you confirm, should data stations like the one listed below be labeled as a Water Level Station or a Tide Prediction station?

image

kvmartin001 commented 2 years ago

@jamesdoyle21 Yeah the color problem is because of the out-of-date fade. Didn't notice on Fri because most stations were current. How's this? image

jamesdoyle21 commented 2 years ago

Much better

cpaternostro commented 2 years ago

The data inventory looks like station 8633532 Tangier Island, VA has been removed. So, YES, it is solely a Tide prediction station. https://tidesandcurrents.noaa.gov/inventory.html?id=8633532 But I'm checking others like station 8630111 JESTERS ISLAND, CHINCOTEAGUE BAY, VA, which is marked as a water levels station, but it is hasn't had data since 1985 so it should be marked as a Tide predictions station.

What is the logic in painting a station yellow (prediction) or red dot (active WL)? How can I help correct this?

--

Chris Paternostro

https://tidesandcurrents.noaa.gov/

On Mon, Apr 18, 2022 at 12:52 PM James Doyle @.***> wrote:

The other thing is there are many stations with Water Level listed as a parameter, but never have data for this.

@cpaternostro https://github.com/cpaternostro can you confirm, should data stations like the one listed below be labeled as a Water Level Station or a Tide Prediction station?

[image: image] https://user-images.githubusercontent.com/82955395/163842758-6b56fb9b-3f9d-4fdd-9f39-e990619bc527.png

— Reply to this email directly, view it on GitHub https://github.com/asascience/ott-coops/issues/434#issuecomment-1101560587, or unsubscribe https://github.com/notifications/unsubscribe-auth/AO77FKMWE5VMQGXULNXTGQTVFWHKRANCNFSM5TLHGO5A . You are receiving this because you were mentioned.Message ID: @.***>

jamesdoyle21 commented 2 years ago

@kvmartin001 see Chris's comment here. Many of the stations we have labeled red (Active WL) have not had water level data here in 30+ years. Correct me if I am wrong here Kevin, but I believe any station that has ever had water level data at one point is labeled red in OceansMap right now. So, @cpaternostro, we would need a list of the co-ops stations that are just tide prediction stations so Kevin can go in and set each one. Let me know if there is an easier way around this.

jamesdoyle21 commented 2 years ago

This is also a backend issue because parameters that haven't had data in 37 years probably shouldn't be showing up in OceansMap..

kvmartin001 commented 2 years ago

@jamesdoyle21 yes, you're right about the labels. I'm labeling stations by the Parameters list, which is in the OceansMap Catalog Item. The parameters come directly from the data service, https://opendap.co-ops.nos.noaa.gov/ioos-dif-sos/ in this case. Maybe our backend needs a filter that removes parameters from the list if they aren't considered current.

As you said, I could also add logic on the frontend that would paint stations based on their ID.

mpiannucci commented 2 years ago

I would really rather not do that @kvmartin001 It should be all driven from the data availabel in the catalog

cpaternostro commented 2 years ago

All historic water level stations should have prediction data available. I guess what the catalog is trying to do is know which stations have available data. Could the list on the left of that Opendap SOS service be the reference of stations that have present day data? Paint the stations in that Stations listings for Water Level red and the rest yellow. STATIONS LISTINGS

--

Chris Paternostro

https://tidesandcurrents.noaa.gov/

On Wed, Apr 20, 2022 at 1:04 PM Matthew Iannucci @.***> wrote:

I would really rather not do that @kvmartin001 https://github.com/kvmartin001 It should be all driven from the data availabel in the catalog

— Reply to this email directly, view it on GitHub https://github.com/asascience/ott-coops/issues/434#issuecomment-1104189994, or unsubscribe https://github.com/notifications/unsubscribe-auth/AO77FKMJ5NMXDFQ6XDO4UJDVGA2KBANCNFSM5TLHGO5A . You are receiving this because you were mentioned.Message ID: @.***>

jamesdoyle21 commented 2 years ago

I am going to leave this issue open. Let's plan to discuss with @cpaternostro on Tuesday

mpiannucci commented 2 years ago

Data catalog should be updated to use get caps to query station availabaility

https://opendap.co-ops.nos.noaa.gov/ioos-dif-sos/SOS?service=SOS&request=GetCapabilities&version=1.0.0

jamesdoyle21 commented 2 years ago

@cpaternostro we now have Tide Prediction stations separated from Water Level stations. Can you take a look at this and let us know your thoughts? Most of the stations labeled Water Level only have tide prediction data because the water level data gives a warning and is grayed out. Curious on your thoughts how to best handle these labels

kvmartin001 commented 2 years ago

We still need to edit Data Catalog to add support for Water Leve active / inactive status.

@mpiannucci On second look here, I'm not seeing any stations tagged as active / inactive in the GetCaps (link above).

There's a DescribeSensor request that looks promising. You can specify a SOS procedure, in this case urn:ioos:network:NOAA.NOS.CO-OPS:WaterLevelActive, and it gives you the list of active water level stations: https://opendap.co-ops.nos.noaa.gov/ioos-dif-sos/SOS?service=SOS&request=DescribeSensor&version=1.0.0&outputFormat=text%2Fxml%3Bsubtype%3D%22sensorML%2F1.0.1%2Fprofiles%2Fioos_sos%2F1.0%22&procedure=urn:ioos:network:NOAA.NOS.CO-OPS:WaterLevelActive

About the DescribeSensor request https://opendap.co-ops.nos.noaa.gov/ioos-dif-sos/ClientGetter?p=3

mpiannucci commented 2 years ago

Are we using opendap or scidap currently for coops data? We should have a very real discussion about whether we keep using scidap for this or hit their opendap directly. For coops OTT, I am partial to hitting them directly. The other versions can use scidap. @kvmartin001 can you do a look through how we do the harvesting and datacatalot stuff currently and see if this could be added to how we do things now?

mpiannucci commented 2 years ago

See our harvester here

https://github.com/asascience/data-harvesters/blob/master/data_harvesters/coops_metadata.json https://github.com/asascience/data-harvesters/blob/master/data_harvesters/harvest_coops.py https://github.com/asascience/data-harvesters/blob/master/data_harvesters/harvest_coops_new.py

And the data_catalog reader:

https://github.com/asascience/data-catalog/blob/master/data_catalog/maracoos/coops.py

kvmartin001 commented 2 years ago

@mpiannucci Yeah I'll take a look. We're currently using SciDAP for CO-OPS simply because our function always tries SciDAP first. Where should the info be stored in the catalog? Maybe we add a "WaterLevelActive" string to the tags list?