API Service Retirement: WaterInsights from WaterNSW Version: 1.0 published on API.NSW

[jonwaland]

Dear API Users,

WaterNSW has introduced another API service that will be providing richer hydrometric data for surface water and groundwater stations across all NSW water sources.

As the result, please be advised that the API – ‘WaterInsights from WaterNSW Version: 1.0’ that is currently published on https://api.nsw.gov.au/ will be retired by 15th September 2023.

We strongly encourage current users of the API – ‘WaterInsights from WaterNSW Version: 1.0’ to transition to accessing the data via our new ‘Water Data API’. You can complete user registration, API subscription and access documentation of the API at https://api-portal.waternsw.com.au/. A quick start guide and user documentation of our new API service are also attached for your reference.

If you require any further assistance in accessing the API, please do not hesitate to reply to this email or contact waterdataservices@waternsw.com.au.

[AnthonyBe]

Shame this repo isn't maintained. I found having the dam info on my Weather Dashboard in HA quite interesting!

[arrikhan]

Same, would like to see it updated....

[arrikhan]

@battlemoose , is there a chance to get this working with new API?

[arrikhan]

For those that want this data and are prepared to do a little bit of work ... I've successfully set this up with REST integration directly in HA. Not as flashy as the wonderful code provided prior ... but this will get it working. I will include where I found my ideas from so you can follow along rather than me explain everything.

1. Register on the new site https://api-portal.waternsw.com.au/ to get an API key.

Once you have a user setup, you can head to Products -> Water Data -> Subscribe. This will activate your account to use the API. From here, head to Profile -> Profile and it will show you your API keys for your subscription. You'll use that in your API call (I stored in secrets file in HA)

2. Build your REST API call using information found at API -> Water Data. The API URL will depend on the type of data you are grabbing as you'll see in the list. My example uses Surface Water Data API to get river height, flow and temperature information. You can build and test your API URL in POSTMAN https://web.postman.co/ to both form the URL with parameters and test output.

https://api.waternsw.com.au/water/surface-water-data-api[?siteId][&frequency][&dataType][&variable][&startDate][&endDate][&dataSince][&pageNumber][&requestId]

Things to consider (from the pain endured): You want to obtain a consistent response so you need to narrow down frequency of data calls based on two things.

• 1. how many API calls you can make a day (29 seems to be limit)
• 2. how many measurements you want to retrieve from different locations
• 3. when the data is sampled/sourced
• 4. how often the data is sampled/sourced (this last one could be overcome if I could be bothered a bit more work but I'm not writing a banking application)

My example checks hourly (~24 calls/day) and samples data between xx:00 (startDate:) and xx:15 (endDate) where it will find 1 set of data for 1 river location! Any longer and you'll get multiple readings which you have to be able to count through. My method counts items in an array to get to data so consistency was key. Time of day matters based on when the data is sampled.

I know all this because I checked [https://waterinsights.waternsw.com.au/] and drilled down to water source, obtained river ID (number in brackets), created an API call in POSTMAN for a 1 hour period and tested. The response showed there were readings every 15 minutes available via API.

Below is an example i used in POSTMAN to test output. You need add a header for your API key in authorization tab with key name Ocp-Apim-Subscription-Key and key value is the API key you obtained earlier (or use -H "Ocp-Apim-Subscription-Key " in a curl command)

https://api.waternsw.com.au/water/surface-water-data-api?siteId=410168&frequency=Instantaneous&datatype=AutoQC&startdate=17-Jun-2024 10:00&endDate=17-Jun-2024 11:00&pageNumber=1

3. Once you have that working, and have identified the URL to use with parameters, setup a REST Integration in HA similar to this one below. You can store in configuration.yaml under "rest: " or in a referenced file if you're broken out your main configuration file into multiple files as i have (rest: reference omitted in below as a result). Also setup your secret for the API key (nsw_water_key for me below).

- resource: https://api.waternsw.com.au/water/surface-water-data-api
  headers:
    Ocp-Apim-Subscription-Key: !secret nsw_water_key
  scan_interval: 3600
  params:
    siteId: 410168
    pageNumber: 1
    frequency: Instantaneous
    datatype: AutoQC
    startDate: "{{ (now() - timedelta(hours = 1)).strftime('%d-%h-%Y %H:00') }}"
    endDate: "{{ (now() - timedelta(hours = 1)).strftime('%d-%h-%Y %H:15') }}"
  sensor:
    - name: "Billabong Downstream Flow Rate"
      unique_id: waternsw_410168_flowrate
      value_template: >
        {% for record in value_json.records %}
          {% if record.variableName == 'FlowRate' %}
            {{ record.value | float }}
          {% endif %}
        {% endfor %}
      unit_of_measurement: "ML/day"
      state_class: measurement
    - name: "Billabong Downstream Water Level"
      unique_id: waternsw_410168_waterlevel
      value_template: >
        {% for record in value_json.records %}
          {% if record.variableName == 'StreamWaterLevel' %}
            {{ record.value | float }}
          {% endif %}
        {% endfor %}
      unit_of_measurement: m
      state_class: measurement
    - name: "Billabong Downstream Water Temperature"
      unique_id: waternsw_410168_watertemperature
      value_template: >
        {% for record in value_json.records %}
          {% if record.variableName == 'WaterTemperature' %}
            {{ record.value | float }}
          {% endif %}
        {% endfor %}
      unit_of_measurement: "°C"
      state_class: measurement

I use this YT video as a guide to test the output from a test curl command to build the template to scrape the data. https://youtu.be/G2YY-HXGmrE. (EDIT) I've since updated the yaml to search for desired variableName and return record.value after seeing a change in output format/data over time.

I also use templates to generate start/end date parameters when it executes. This is based on examples in HA REST docs page.

Once you've written the file, restart HA to generate sensors and start polling.

Last NOTE: I lost a day rebooting regularly to get this working which polled for data which worked, but my sensor config didn't so i ran out of calls and had to wait another day to get it right. You can check how many polls you've made good or bad on the api website.

Good luck!