OPP-6: WIS Djibouti

[rogers492]

OPP-6: WIS Djibouti (working title)

1. Introduction

   1. The OpenWIS Association hosts multiple open source software development projects that support or enable the exchange of global meteorological information.
   2. We use the GitHub platform for our collaborative development. The rules and expectations for participating in an OpenWIS® Project are defined in the Technical Rules and the Code of Conduct.
   3. The basic governance of the OpenWIS Association is laid out in the Articles of Association and the Internal Rules.

2. Purpose

   1. This charter establishes the foundation for the project team to begin developing, through a pilot project, the next iteration of the OpenWIS core software, by outlining the scope of this pilot project, identifying the contributing members and defining their responsibility.
   2. Scope of Authority This task team is charged with conducting a pilot study on the future WMO Information System, often referred to as WIS 2.0.
   3. Authority The authority for this project comes from the OpenWIS Steering Committee (SC), on recommendations from the OpenWIS TC. Outcomes from this and other pilot studies provide insight into WIS 2.0 specifications, and may ultimately factor into an architectural design and overall OpenWIS 5 plan with timelines.

3. Pilot Project Overview Overview of pilot study task.

   1. Pilot Project Statement of Need or User Stories
      1. Imagine a WIS based on a set of common standards layered on top of each other and in which all data is stored in the Cloud as static files,
         • eg: TCP/IP, HTTP, URL.
      2. Imagine that access to this WIS is easy, even on basic infrastructure,
         • eg: internet only.
      3. Imagine that finding the data you want through common commercial search engines is simple and does not rely on a deep understanding of complex metadata,
         • eg: because the metadata is stored as tags in human readable web-pages and is routinely indexed by web crawlers.
      4. Imagine that making new static file data available through WIS and ensuring that it's simple metadata gets indexed automatically is easy, even on basic infrastructure.
      5. Imagine that, once you have found the data you are looking for, subscribing to updates or routine delivery of that data is easy.
      6. Imagine that you have to demonstrate the concept of such a WIS to the TECO conference in early 2018 and that you have to easily convince the audience of the benefits such a WIS would bring.
   2. Pilot Project Scope
      1. Create a demonstration prototype that is suitable for presentation at TECO 2018 by a member of the SC.
      2. The Project team is responsible for:
         • Development of the demonstration prototype software
         • Delivery of a package that will be suitable for demonstration at TECO
         • Any documentation or training required to deliver the demonstration
         • Any supporting infrastructure required to do the demonstration
         • Provision of realistic data for the demonstration.
      3. Pilot Project Goals
         • Demonstrate aspects of the future WIS 2.0 concept
         • Gain support at TECO for the whole WIS 2.0 concept, or at least significant aspects of it.
      4. Pilot Project Deliverables
         • Provide a bulletized list of expected deliverables for the Project Task:
         • TBD
         • TBD

4. Roles and Responsibilities:

Resource Name	Organization	Role	Responsibility	Time period resource needed for
Resource_A	Org_A	Role_A	Responsibility_A	Period_A
Resource_B	Org_B	Role_B	Responsibility_B	Period_B

5. Milestones & Schedule

Project Milestone	Begin Date	End Date
Milestone 1	YYYY/MM/DD	YYYY/MM/DD
Milestone 2	YYYY/MM/DD	YYYY/MM/DD

6. Configuration Management Process

   • Define process for checking in/out code associated with pilot study

7. Communication and Reporting

   • Frequency and mode of team meetings, squad/team names, GitHub IDs etc
   • Define interactions with PMC including reports at PMC meetings
   • Define interactions with TC
     • Final report/briefing on pilot study findings and recommendations

[rogers492]

From @tg4444 :+1: Subject: Concerning Djibouti

We have already began investigating the AWS features required for the Djibouti pilot implementation, but before we delve any further, I want to make sure that our understanding is correct.

From the issue description - and based on my current understanding of OpenWIS - I came up with the following system requirements. Please correct/enrich this list:

• R1. All data/metadata storage and processing occurs on AWS/RDS
• R2. There exists a back-end user interface for the Djibouti administrator to upload new data/metadata
• R3. No data/metadata editing will occur
• R4. Once the metadata is uploaded to the system, an HTML page is dynamically generated (by using an XSD maybe?) and that page will be available to search engines.
• R5. The HTML page mentioned above will also contain a link that will forward users to another page, in which they will be able to subscribe to data, or manually download it.
• R6. Subscription dissemination will take place though FTP and email.
• R7. The data for subscriptions will be forwarded to the end-users as soon as they are uploaded by the Djibouti administrator (i.e. no time-based subscriptions).

Apart from the list of requirements, there are some questions I would like to ask regarding this pilot. Q1. As far as I know, there is no location information (i.e. Country/Region/City) within the metadata, except from the longitude/latitude bounding box. If this is the case, how would someone wanting to find metadata for UK, Exeter be able to do so, based on Google, without knowing the exact "ID" of the metadata? Q2. Even if we were able to provide location information, if someone wanted to subscribe to the whole UK data, he would need to click through 30 thousand pages in order to do so. Is this the desired functionality? Q3. In order to provoke search engines into indexing and ranking our dynamically-generated HTML pages, we need to link towards them from other web-pages. An old common practice was to create a static web-page with hidden links to the dynamic pages, and have google go through that one. Search engines are now smart enough to ignore such pages. Do you have any preference on where the dynamic pages could be linked, or would you like us to research options? Q4. Even with the dynamic pages being indexed, unless their content differs substantially, google will likely only show one result, and add a link at the bottom saying something like "There are 150,000 omitted results similar to this one. Click here if you want to search again including those results". If we want to make these pages look more different, a greater percentage of custom content should be displayed. Do you have any preference on where the dynamic pages could be linked, or would you like us to research options?

[6a6d74]

Hi - I've tried to put a narrative in place around this project. See below. I've started from the perspective of super-low tech, but we can evolve as we see fit.

@tg4444 : I will try to answer your questions tomorrow...

---

​ OpenWIS Djibouti

[see Spatial Data on the Web Best Practices for recommended implementation patterns - https://www.w3.org/TR/sdw-bp/]

Mouktar is responsible for recording weather observations in the Tadjourah region of Djibouti. Data is collected for a number of locations:

• Tadjoura Port
• Lac Assal
• Dorra
• Assa Gaïla

Each day, observation data is sent to Mouktar via SMS message. Mouktar enters the information for each location into a separate tabular data file (CSV format) and copies the updated files to his Web server to publish the data online.

[note: alternatively, these CSV files may be pushed to AWS S3 to achieve the same outcome] Mouktar has been collating this data for several years and has created a valuable timeseries of observation data for these locations.

[note: it would be useful and illustrative to include some GUI components to interact with the CSV content.]

Each of the four published CSV files are datasets that are registered on the WIS - indicating that these are authoritative datasets provided with the approval of Djibouti's Permanent Representative to WMO.

Mouktar has created a Web page describing each dataset - providing metadata in a human readable form. Mouktar has specified that this data is available as "Essential" data under WMO data policy: openly accessible and free to use without restriction. Each Web page also contains structured markup (embedded JSON-LD Githubissues.

Githubissues is a development platform for aggregating issues.