Outline product ecosystem

[zacharlie]

Outline existing tools, frameworks, utilities, and processes that may be used alongside #1 to determine product and process structure

[zacharlie]

A number of existing tools and frameworks exist providing similar functionality to at least some of the components which form the core application and product framework.

• NASA has the Earthdata portal, as is Copernicus hub, although many of these larger open science data repositories focus on Earth Observation information. Some data source platforms offer frontends or desktop applications for clipping or processing data, but this is generally implementation specific
• Awesome Geodata is simply a curated list of spatial data sources and related resources. There are a ton of similar lists, applications, and platforms (such as custom ckan implementations) which attempt to provide access to data. Very often these collections of data are agency specific, poorly maintained, are of low quality, or simply lack the proper incentive to ensure that they maintain high standards
• G3W Suite and Lizmap provide web interfaces with support for users and some level of access to backend processing APIs, but primarily function as either a Geodata CMS or a web frontend
• The ArcGIS HUB, AGOL, and "Living Atlas" attempt to provide comprehensive proprietary data access platform for various sources
• Other proprietary platforms offer some form of data cataloguing and/ or data clipping and downloading utilities, such as the cdngiportal
• PanGeo/ ODC and platforms like digital earth africa provide bigdata access using modern technologies like STAC, but are typically designed for EO and raster analytics and may rely on significant infrastructure not available to typical users or industries
• HOT export tool/ imposm allow downloading and importing OpenStreetMap Data, whilst Geofabrik provides data extracts by region. The Overpass API allows limited downloading of OSM data. Some platforms like healthsites.io aim to enrich and support application specific outputs with OSM data. Additional sources for OSM are available at openstreetmap.org/export
• GeoNetwork/ GeoNode/ CKAN provide metadata catalogues and SDInfrastructure for searching and publishing data. Some plugin support might be available for extending such catalogues with spatial processing/ sourcing/ clipping capabilities. These platforms are designed primarily as catalogues, and may be overkill for a simple system where vendors define their own
• Mapshaper or placemark provide some examples of online data manipulation tools
• Desktop tools and web platforms provide analysis, libraries like turfjs might provide client side analysis for interfaces like OL and leaflet, whilst plugins (e.g. QGIS SRTM Downloader) might provide data access for specific stores of data. Any combination of these tools is typically used for an application specific function.

Understanding this ecosystem allows us to identify and highlight the project goals and general mission.

From the outline above, it is clear that:

• There is no clear single source of truth for sourcing spatial data
• Most applications and platforms are specifically designed for a purpose other than to allow users to access a collection of data of their own choosing only for a particular area of interest

The GeoData Mart attempts to solve some of the issues to some of these projects and products by providing additional value in pursuing the following goals:

• Providing a single platform for simple and integrated access to various spatial datastores
• Making updated high quality spatial data available to everyone as easily as possible
• Providing an integrated and accessible data catalogue which may be used to retrieve data directly from other GIS systems, in much the same way QuickOSM is able to do for QGIS and OpenStreetMap
• Allowing vendors fine grained control of their data
• Foster innovation and ensure that users spend more time doing meaningful analysis than redundant data processing
• Providing an abstraction for a data elements metadata within the catalogue to be referenced with standards compliant APIs
• Providing an abstraction for consuming and processing external data sources via standards compliant APIs
• Providing an open source platform which can be leveraged by third party data vendors to supply users with data simply and efficiently, and provide additional market value for existing datastores
• Providing a platform that supports the proper metering and billing of data usage in an effective way that promotes sustainability and provides the proper incentives to data vendors to ensure that their data is listed, available, and up to date.
• Providing a platform that would allow public and private agencies (such as National Governments) to host internally and support open access to their data stores
• Promoting Open Source, Open Data, Open Science, Transparency, and respecting user privacy and security
• Being awesome