Extracting raw data based on polygon input

[ramyaragupathy]

An API endpoint, where the user can extract a certain OSM feature type for a specific hashtag within the input polygon and the timeframe that the user has supplied. Eg: buildings with #hotosm-project-1234 on 7th Dec 2021

Available only for permissioned users since we share user metadata on features and tasking manager statistics.

Endpoint: /rawdata POST

Source:

Insights - so that data always remain consistent over a period of time

Sample Request Body:

{
“feature_type”: [“building”,”highway”, “landuse”, “waterway”],
“Hashtag”: [“hotosm-project-1234”,”missingmaps”],
“From_timestamp”: “2021-08-27 09:00:00”,
 “to_timestamp”:”2021-08-31 09:00:00”,
  “Output_type”: GeoJSON/KML/SHP
 "geometry": {
        "type": "Polygon",
        "coordinates": [
          [
            [
              71.08154296875,
              34.125447565116126
            ],
            [
              71.83959960937499,
              34.125447565116126
            ],
            [
              71.83959960937499,
              34.615126683462194
            ],
            [
              71.08154296875,
              34.615126683462194
            ],
            [
              71.08154296875,
              34.125447565116126
            ]
          ]
        ]
      }
}

Acceptance Criteria:

• Geometry
  • Required field
  • Can be a polygon or a multipolygon
  • Polygon size limited to 100sq.km - Tasking Manager threshold is 5000 sq.km
• Feature Type:
  • Refers to OSM keys
  • It is not a required field
  • existence of building tags with values other than no/false/0/null
  • existence of highway tags with values other than no/false/0/null
  • existence of landuse tags with values other than no/false/0/null
  • existence of waterway tags with values other than no/false/0/null
  • Array field with text values
  • Validation
    • Trim for spaces
    • No special characters and numbers that are not allowed in OSM key - only : and _ supported, any characters or numbers other than that throws an 402 error
• Hashtag
  • Not a required field
  • Array field with text values
  • Validation
    • Trim for spaces
    • Check for Single character - alphanumeric and special chars and throw an error message, at least 2 characters required
    • Special characters not supported https://github.com/hotosm/galaxy-api/blob/master/src/galaxy/validation/models.py#L40
• Timestamps
  • Not a required field
  • UTC format
• Output Type:
  • GeoJSON
  • KML
  • SHP
  • Compressed format: gz, zip, bz2 - @itskshitiz321 to research and choose the best algorithm

[ramyaragupathy]

cc @robsavoye @omranlm @JorgeMartinezG

[ramyaragupathy]

The current data from Insights is a historical snapshot for the given time period. So instead of using raw-data endpoint let's rename it to historical-snapshot.

@itskshitiz321 @omranlm @JorgeMartinezG @robsavoye

[kshitijrajsharma]

We can keep rawdata as prefix to make it more clearer ? like this , rawdata/historical-snapshot/ , so that when we provide current snapshot we can say , rawdata/current-snapshot @ramyaragupathy What do you think ?

[ramyaragupathy]

sounds good @itskshitiz321