Belize SURFACE CDMS

[isedwards]

@scottylad876 is from the National Meteorological Service of Belize.

Since 2016, they have being working on a web based CDMS project. This was originally written in PHP but is now being redeveloped using Python to take advantage of the wide range of scientific libraries.

Over the next few days I will create new conversations for each of the following topics:

• OpenCDMS progress, current project structure and possible ways of collaborating
• Compliance testing and ensuring CDMSs meet the WMO CDMS Specifications no.1131
• Belize suRFACE CDMS technology choices and similatities in approach

Dwayne, could you see whether other members of the team in Belize are happy to join GitHub and be part of these discussions?

[scottylad876]

@isedwards

Thank you for including us. We have our lead developer Fabio Sato based in Brazil, and we have our in house developer who is part of the NMS Belize, Shanika Matura. Could you please include these two persons as part of the discussions. I am looking forward to the three topics you proposed because these are the same topics we would like to have to resolved on our end. I think i sent you an overview of what we have been doing but if not see it attached. Overview of CDMS in Belize (1).pdf

[isedwards]

I've sent invitations to @fabiosato and Shanika Matura's Hydromet email address . When they have both joined we can create new issues to discuss compliance testing and OpenCDMS progress.

Is HydrometDB still in use or have you moved over completely to using SURFACE?

[scottylad876]

We are still using HydrometDB but some of the limitations of the system started showing up last year. We are currently in the process of loading all the data from HydrometDB into Surface so that we can begin using it in day to day operations withing the next 3 weeks.

Allow me to give you a bit of background to what we are doing and how Surface fits in. Our vision for years has been to have a system that would be able to ingest a wide range of data, process this data and make it available to our staff and our stake holders. We also needed this system to be able to update our website particularly with our AWS data. Presently we operate 48 AWS, 20-22 Conventional Manual Stations, and 22 HOBO automatic(manual download) rain gauges. Map 95% of our AWSs store data every 5mins(i.e. rainfall, max, min etc), about 3 store data every hour, and another 3 store data every 15mins. So its a lot of data to be ingested, processed and made available in realtime on our website. HydrometDB allowed us to do this. Particularly our remote satellite stations, we used NOAA's LRGS to download our DCP messages, store them in HydrometDB.

So with HydrometDB, we created two instances of the system at our office by making use of Postgres DB replicating feature. The Primary system was used to ingest data and used internally for our staff, while the secondary system(read only) was updated instantly once changes were made to the primary. This secondary system was used by our stakeholders externally. Last year we began to see the limitations of not having a timeseries database and we started having performance issues, and so the upgrade to Surface came at the perfect time.

With Surface we created user stories for each part of the main parts of the system. We had 5 basic parts;

• Station Management - everything to do with monitoring and management of the weather station
• Data - import, export, data inventory
• Outputs - charts, reports, spatial analysis etc
• Quality Control - Automatic and Manual QC procedures
• User Management

We have been working to complete these over time. The WMO 1131 has been in a sense guiding us, but that document is so extensive its virtually impossible for us to properly assess what we doing, and again if others are working on the same thing why do this on our own.

So this is where we are now.

[isedwards]

One of the things that really interests me about Surface is that you have made the same technology choices that we have been discussing (Python, PostgreSQL/PostGIS, database replication, Django/Flask etc.)

Do you have any thoughts on whether you would be happy to share the Surface source code on GitHub as open source (for example the Climsoft Desktop software has been online since 2015 under a GPL v3 licence).

[fabiosato]

I think we are ok with sharing the source code on GitHub under an open source license to cover non-profit applications, what do you think?

Regarding the tech stack choices I would be happy to schedule a call to discuss about it.

[isedwards]

Sounds good. Give me a couple of weeks to finish off a some urgent work, I'll email in early March to see if we can set up a call.

[scottylad876]

Ian remember to let us know when you are available for a call