Investigate and fix the BoM Water Online SOS2 services

[AnaBelgun]

Currently the "Watercourse levels" and "Watercourse Discharge" are showing data up to 1 Jan 2016, and 1 Jan 2017 respectively. BoM is keeping the SOS2 services up to date with data being harvested as late as 22-05-2018.

Screen grab of data viz on NEII Viewer and its url: http://www.neii.gov.au/viewer/#share=s-raLuQhkqOIegZ6PUPpyzR9Ik8YX

and screen grab of the SOS2 service with the data stamp and its url: http://www.bom.gov.au/waterdata/services?service=SOS&version=2.0&request=Getobservation&featureofinterest=http://bom.gov.au/waterdata/services/stations/410001&procedure=http://bom.gov.au/waterdata/services/tstypes/Pat4_C_B_1_DailyMean

[meh9]

Watercourse Discharge seems fine and has reasonably current data up to 2018-05-22 (2 days ago), so I think that layer is fine. Note that the layer definition only has a start date:

https://github.com/TerriaJS/neii-viewer/blob/master/datasources/includes/water-items.ejs#L673

Watercourse Levels only has data up to 2016. Looking at the layer definition that's because for some reason it has an end date specified in the config: http://neii.gov.au/viewer/init/neii-water.json

However, this end date was actually removed a year and a half ago, maybe there hasn't been a release since then?

https://github.com/TerriaJS/neii-viewer/commit/134594c96208762bd0380c1fefcb984d626a9f03#diff-d6f97544c2e0c83319cf456836a7f683

[AnaBelgun]

Follow up with BoM again

[ChantelleDoan]

Hi Ana,

Apologies for this issue and the impact on your testing. I am touching based with the WDO team on this issue.

I have the below notes from Kisters developers who supports the WDO WISKI and SOS2.

Jo Sullivan who is our contact with Kisters is away until next week. I will continue to touch base with Jo to track the progress of this issue.

July 11th: However I am encountering the issues you described with SOS2 requests timing out. This seems to be fairly inconsistent, as some requests work and others don't. Even the entry point getCapabilities call does not work for me at all www.bom.gov.au/waterdata/services?service=SOS&version=2.0&request=GetCapabilities

But this call accessing specific data seems to always work for me: http://www.bom.gov.au/waterdata/services?service=SOS&version=2.0&request=Getobservation&featureofinterest=340000,41001701,G8150036,G8155558,G8110012,G0060009,G0060046,G0280004,G0290228,G8140008&observedProperty=Water%20Course%20Level,Water%20Course%20Discharge&procedure=Pat3_C_B_1,Pat4_C_B_1&temporalFilter=om:phenomenonTime,2017-9-18T00:00:00Z/2017-9-25T00:00:00Z

I think it's strange that the KiQS calls are all returning fine, but the SOS2 calls have problems. To me this indicates that the issue is not likely to be with KiWIS. If there was a problem with KiWIS then I would expect there to be an issue with the KiQS requests as well. I will double check that with our developers before Thursday. Regards, Chantelle Doan | Data Services, NEII Programme Manager [Description: Description: cid:389443803@21082013-35A7] Digital & Data Group, Data Program Bureau of Meteorology GPO Box 1289 Melbourne VIC 3001 Level 7East, 700 Collins Street, Docklands VIC 3008 Tel: +61 3 9669 4475 | chantelle.doan@bom.gov.aumailto:chantelle.doan@bom.gov.au | www.bom.gov.auhttp://www.bom.gov.au/

From: AnaBelgun notifications@github.com Sent: Monday, 23 July 2018 5:11 PM To: TerriaJS/neii-viewer neii-viewer@noreply.github.com Cc: Subscribed subscribed@noreply.github.com Subject: Re: [TerriaJS/neii-viewer] Investigate the BoM Water Online SOS2 services (#106)

Follow up with BoM again

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/TerriaJS/neii-viewer/issues/106#issuecomment-406961757, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ARD-l4Ij9PJktsXgWcTRVCo2fGRq3EvSks5uJXcbgaJpZM4ULVSt.

[AnaBelgun]

Thanks @ChantelleDoan we're getting closer to testing the WCS protoypes (service and front end)

It might be connected with this old issue: https://github.com/TerriaJS/neii-viewer/issues/97#issuecomment-315971187

[AnaBelgun]

The investigation is completed and BoM is aware of the issue and trying to fix it. Will close it and open a new one, once BoM finalises the work.

[AnaBelgun]

25 Oct comment: I've looked at your SOS2 call and have a couple of questions/suggestions we could look at.

I don't have the full background of your requirement for this call, but what is the purpose of calling for all these different timeseries types?

As I understood from your emails, all you're after is the list of sites with "water course level" data. You should effectively be able to remove all the "" lines out of your list the "GetFeatureOfInterest" call will filter on as the call "" will give you this list.

In fact, the way our system is set up, if there is an observed parameter for a station, it automatically has all these time series types rolled out, regardless of whether there is any data. It's also pretty rare to have an observed parameter but no data as the system setup is driven by the incoming data.

Also, the "" is not an option for the "GetFeatureOfInterest" call and gets ignored in the query.

If you do want this feature (filtering for data since 1980), then you'll have to use the "GetDataAvailability" call, use the "ObservedProperty" to extract all water course level information and do some postprocessing to filter the list on "beginPosition" in the "phenominonTime – TimePeriod" output.

I hope this helps and please get back to me if you have further questions/issues.

Regards, Rudi van den Bos

[AnaBelgun]

25 Oct follow up: Just following up to say we have implemented the suggestion to remove the "" lines from GetFeatureOfInterest calls made to BoM servers, and that seems to solve the problem.

[AnaBelgun]

changes rolled out and updated NEII to latest TerriaJS version on Monday 5 Nov on:

master

viewer.neii.gov.au and staging.neiiviewer.nicta.com.au

catalogue-staging

catalogue-staging.neiiviewer.nicta.com.au

[AnaBelgun]

@stevage @meh9 To test in IE too - tested. ok It seems to be an error: Great news re getting SOS2 fixed on the viewer. Just one thing, has it been tested on Internet Explorer? It might just be my particular implementation, but it crashed IE when I went to the site. We often have issues with WDO & other websites when we deploy to IE, it is a hard browser to please. All is good on Chrome,