As a developer, I want to investigate, understand, and implement a reader for the WRDS HEFS service

[epag]

---

Author Name: Hank (Hank) Original Redmine Issue: 121930, https://vlab.noaa.gov/redmine/issues/121930 Original Date: 2023-10-26

---

The URL is:

https://nwcal-wrds.[domain]/api/hefs/v1/

Swagger:

https://nwcal-wrds.[domain]/api/hefs/swagger/

I would like to investigate the API and gain an understanding relatively soon. We can then decide if and when we want to write a reader for it. Marking this as high priority, but not urgent,

Hank

[epag]

---

Original Redmine Comment Author Name: Hank (Hank) Original Date: 2023-10-26T15:58:33Z

---

Fixed a typo.

[epag]

---

Original Redmine Comment Author Name: Hank (Hank) Original Date: 2023-11-02T16:56:56Z

---

We are waiting on this until a user asks for it, but doing an initial investigation sooner makes sense.

This might end up becoming a framework picked up for the other services down the road.

Hank

[epag]

---

Original Redmine Comment Author Name: Hank (Hank) Original Date: 2023-11-02T17:37:41Z

---

Here is an example URL:

https://nwcal-wrds.[domain]/api/hefs/v1/series/?limit=10&start_date=2023-10-01

Here is the example output:

{
    "count": 6895131,
    "next": "http://nwcal-wrds.[domain]/api/hefs/v1/series/?limit=10&offset=10&start_date=2023-10-01",
    "previous": null,
    "results": [
        {
            "id": "b939ffe0-a3e4-4dd6-9c0d-d2f325dbf874",
            "created_at": "2023-10-11T18:57:48.449916Z",
            "updated_at": "2023-10-11T18:57:48.449940Z",
            "deleted_at": null,
            "is_deleted": false,
            "type": "instantaneous",
            "location_id": "PDLC2",
            "parameter_id": "QINE",
            "ensemble_id": "MEFP",
            "ensemble_member_index": 1951,
            "time_step_unit": "second",
            "time_step_multiplier": 21600,
            "start_date_date": "2022-05-16",
            "start_date_time": "00:00:00",
            "end_date_date": "2022-06-15",
            "end_date_time": "00:00:00",
            "forecast_date_date": "2022-05-16",
            "forecast_date_time": "00:00:00",
            "miss_val": "nan",
            "station_name": "PDLC2R",
            "lat": 38.4872222222,
            "lon": -105.373055556,
            "x": null,
            "y": null,
            "z": null,
            "units": "CFS",
            "creation_date": "2022-05-16",
            "creation_time": null
        },
        {
            "id": "dc822be7-a136-4845-b472-00f65d5cf7fa",
            "created_at": "2023-10-11T18:57:48.498368Z",
            "updated_at": "2023-10-11T18:57:48.498377Z",
            "deleted_at": null,
            "is_deleted": false,
            "type": "instantaneous",
            "location_id": "PDLC2",
            "parameter_id": "QINE",
            "ensemble_id": "MEFP",
            "ensemble_member_index": 1952,
            "time_step_unit": "second",
            "time_step_multiplier": 21600,
            "start_date_date": "2022-05-16",
            "start_date_time": "00:00:00",
            "end_date_date": "2022-06-15",
            "end_date_time": "00:00:00",
            "forecast_date_date": "2022-05-16",
            "forecast_date_time": "00:00:00",
            "miss_val": "nan",
            "station_name": "PDLC2R",
            "lat": 38.4872222222,
            "lon": -105.373055556,
            "x": null,
            "y": null,
            "z": null,
            "units": "CFS",
            "creation_date": "2022-05-16",
            "creation_time": null
        },
</code>

Its providing only metadata, and my URL limited it to the first 10 headers.

Let me take a look at the Swagger,

Hank

[epag]

---

Original Redmine Comment Author Name: Hank (Hank) Original Date: 2023-11-02T17:47:21Z

---

Not a big fan of the Swagger documentation. It may not be fully baked. I've asked David Moring to provide an example URL for a specific scenario that I can use as a starting point for understanding the service.

Thanks,

Hank

[epag]

---

Original Redmine Comment Author Name: Hank (Hank) Original Date: 2023-11-02T17:47:39Z

---

Waiting for a reply,

Hank

[epag]

---

Original Redmine Comment Author Name: Hank (Hank) Original Date: 2023-11-02T18:51:28Z

---

Was told in the WRES/WRDS meeting that they will work that request into examples dovetailed into a Jupyter Notebook they are hoping to release next Wednesday. I also informed them about the lack of a time zone in the response, so that we can't be sure in what time zone dates are presented.

Last point... They intend to move all services to something like the HEFS approach, with the next one likely being the Location Service, which will be moved to the Cloud.

Leaving on hold,

Hank

[epag]

---

Original Redmine Comment Author Name: Hank (Hank) Original Date: 2023-11-16T21:01:53Z

---

Here is the Jupyter Notebook (enter the token "WRDSJupyter"):

https://nwcal-wrds.[host]/jupyter/lab/tree/hefs

It should help to investigate how to use the HEFS service. We would then need to write a brand new reader to prepare proper requests and parse the results.

Hank

[HankHerr-NOAA]

Latest from Gautam chatted to me today (edited to clean it up)...

HEFS is up and running in UAT. This is going to be our new landing page in the cloud (not pushed up to the cloud yet but the links work):

https://nwcal-wrds-dev01.[domain omitted]/.

heck out this notebook. it shows how to retrieve an ensemble for a location:

https://colab.research.google.com/github/NOAA-OWP/data-service-notebooks/blob/master/HEFS/2_ensemble_plotting_demo.ipynb

I don't think we need to code against it, yet, but we should at least take a look before the next WRES/WRDS meeting (early November, since I canceled the meet tomorrow) so that we can discuss it if we have questions.

Thanks,

Hank