search

UGA-ITOSHumanitarianGIS / CODV2API

COD Services for Common Operational Datasets used by Humanitarian Operations -API, Documentation, Progress and Issue tracking and Resources
Apache License 2.0
6 stars 0 forks source link

Help - Structure of API call to extract a geojson with all admin2 geojson for a specific country #24

Open Edouard-Legoupil opened 1 year ago

Edouard-Legoupil commented 1 year ago

Hello -

I am looking at the documentation - https://apps.itos.uga.edu/CODV2API/Help - and I am not clear on how to build the API call to extract the a geojson based on country name

I would assume - based on - https://apps.itos.uga.edu/CODV2API/Help/Api/GET-api-v1-themes-cod-ab-locations-pCode-versions-current-format-level

{pCode} is the country for instance GTM {format} should be geojson - or maybe json {level} is the admin level - aka 2 https://beta.itos.uga.edu/CODV2API/api/v1/cod-ab/locations/{pCode}/versions/current/{format}/{level} tried different approach for different countries but none is working -

https://beta.itos.uga.edu/CODV2API/api/v1/cod-ab/locations/HTI/versions/current/json/2 https://beta.itos.uga.edu/CODV2API/api/v1/cod-ab/locations/HTI/versions/current/geojson/2

https://beta.itos.uga.edu/CODV2API/api/v1/cod-ab/locations/GTM/versions/current/json/2 https://beta.itos.uga.edu/CODV2API/api/v1/cod-ab/locations/GTM/versions/current/geojson/2

My goal is to build a simple R function to pull the data from the API - see below -

`

' Get admin2 polygons from API

'

' Queries the COD API sever to return admin2 geometry and pcode for a specific

' country based on iso name.

' As the resulting data is expected to be used in a shinyApp, the geomtry is simplified

'

' @param ISO3 ISO3 code of country

' @param simplify Logical: whether to simplify or not

' @param dTolerance parameter passed to [sf::st_simplify()]

' @importFrom sf st_read st_simplify

' @return sf object

' @export

f_get_admin2_boundaries <- function(ISO3, simplify = TRUE, dTolerance = 500){

stopifnot(ISO3 %in% country_codes$ISO3)

generate query string

server <- "https://beta.itos.uga.edu/CODV2API/api/v1/cod-ab/locations/" parameter <- "/versions/current/geojson/2" api_query <- paste0( server, ISO3, parameter)

read and create feature table

df_geom <- sf::st_read(api_query)

if(any(is.na(df_geom$adm2_source_code))){ warning("NAs found in Admin2 codes...") }

if(simplify){ sf::st_simplify(df_geom, preserveTopology = TRUE, dTolerance = dTolerance) } else { df_geom }

}

`

Can you kindly help to get that function working?

Thanks, Edouard

UGA-ITOSHumanitarianGIS commented 1 year ago

Hello! Try the following example for Burkina Faso level 1 in geoJSON format.

https://apps.itos.uga.edu/codv2api/api/v1/themes/cod-ab/locations/BFA/versions/current/geoJSON/1

ISO-3 may be swapped for another available country: https://codgis.itos.uga.edu/arcgis/rest/services

From: Edouard Legoupil @.> Sent: Monday, July 3, 2023 3:08 PM To: UGA-ITOSHumanitarianGIS/CODV2API @.> Cc: Subscribed @.***> Subject: [UGA-ITOSHumanitarianGIS/CODV2API] Help - Structure of API call to extract a geojson with all admin2 geojson for a specific country (Issue #24)

[EXTERNAL SENDER - PROCEED CAUTIOUSLY]

Hello -

I am looking at the documentation - https://apps.itos.uga.edu/CODV2API/Help - and I am not clear on how to build the API call to extract the a geojson based on country name

I would assume - based on - https://apps.itos.uga.edu/CODV2API/Help/Api/GET-api-v1-themes-cod-ab-locations-pCode-versions-current-format-level

{pCode} is the country for instance GTM {format} should be geojson - or maybe json {level} is the admin level - aka 2 https://beta.itos.uga.edu/CODV2API/api/v1/cod-ab/locations/{pCode}/versions/current/{format}/{level} tried different approach for different countries but none is working -

https://beta.itos.uga.edu/CODV2API/api/v1/cod-ab/locations/HTI/versions/current/json/2 https://beta.itos.uga.edu/CODV2API/api/v1/cod-ab/locations/HTI/versions/current/geojson/2

https://beta.itos.uga.edu/CODV2API/api/v1/cod-ab/locations/GTM/versions/current/json/2 https://beta.itos.uga.edu/CODV2API/api/v1/cod-ab/locations/GTM/versions/current/geojson/2

My goal is to build a simple R function to pull the data from the API - see below -

`

' Get admin2 polygons from API

'

' Queries the COD API sever to return admin2 geometry and pcode for a specific

' country based on iso name.

' As the resulting data is expected to be used in a shinyApp, the geomtry is simplified

'

' @paramhttps://github.com/param ISO3 ISO3 code of country

' @paramhttps://github.com/param simplify Logical: whether to simplify or not

' @paramhttps://github.com/param dTolerance parameter passed to [sf::st_simplify()]

' @importFrom sf st_read st_simplify

' @returnhttps://github.com/return sf object

' @exporthttps://github.com/export

f_get_admin2_boundaries <- function(ISO3, simplify = TRUE, dTolerance = 500){

stopifnot(ISO3 %in% country_codes$ISO3)

generate query string

server <- "https://beta.itos.uga.edu/CODV2API/api/v1/cod-ab/locations/" parameter <- "/versions/current/geojson/2" api_query <- paste0( server, ISO3, parameter)

read and create feature table

df_geom <- sf::st_read(api_query)

if(any(is.na(df_geom$adm2_source_code))){ warning("NAs found in Admin2 codes...") }

if(simplify){ sf::st_simplify(df_geom, preserveTopology = TRUE, dTolerance = dTolerance) } else { df_geom }

}

`

Can you kindly help to get that function working?

Thanks, Edouard

— Reply to this email directly, view it on GitHubhttps://github.com/UGA-ITOSHumanitarianGIS/CODV2API/issues/24, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AME2U5UB2ACKIHADQNHSJTTXOMKDJANCNFSM6AAAAAAZ4Z5V64. You are receiving this because you are subscribed to this thread.Message ID: @.***>

fititnt commented 1 year ago

@Edouard-Legoupil I think you might be interested in the index page, which lists what's available. Really worth it, even if just to help user get a general idea repl or quick debug:

curl -X 'GET' \
  'https://beta.itos.uga.edu/CODV2API/api/v1/locations' \
  -H 'accept: application/json'

Also, likely some endpoints not up to date (on at least the documentation is better use the https://apps.itos.uga.edu/CODV2API/Help, but an early version of the CODV2API with OpenAPI3 is available https://hapi.etica.ai/eng-Latn/api/UN/common-operational-datasets-api/ (https://hapi.etica.ai/api/UN/CODV2API/eng/openapi.yaml).

And even if UNHCR might use different population statistics, the CODV2API both have population statistics and a variant of country borders, the edge matched.

By the way, I did not finished better integration with another open source solution, Orange Data Mining (here an preview https://github.com/fititnt/orange3-hxl) BUT on Orange, with https://github.com/biolab/orange3-geo/issues/149 fixed (likely replaced by an COD version, so every production would only need the data, not also download the geometries) could be another app to be used. The current use on Orange is not bad (it uses Natural Earth, but obviously does not have P-Codes) but in theory would be possible to allow very nice data plotting (and the Orange workflows are somewhat reusable, just change the data source).

But in general, the CODV2API is very usable! In my personal option, it might be worth doing a local cache of files and if requesting several countries at once, do it in sequence (not parallel requests). Just a single computer might not be a problem, but let's say you have several installations, everyone downloading again and again, maybe at some point the user may have a timeout. Also, it could be a good idea if the user agent has a hint on how to contact the developer (e.g you from your app) and the app version, so in the worst case scenario, UGA-ITOS could check server logs.

galalH commented 1 year ago

You can use esri2sf for that. Find the relevant MapServer from the directory and, say, you want the ADM2 boundaries in Mexico, you can just call esri2sf::esri2sf("https://codgis.itos.uga.edu/arcgis/rest/services/COD_External/MEX_pcode/FeatureServer/2").

UGA-ITOSHumanitarianGIS commented 1 year ago

Will check this out. Thanks for suggesting this option. Thanks @fititnt for suggesting the cache! The server environment is now ArcGIS 10.91 There may be optimization for larger admin 2 content (or admin3 and rare admin 4 availability) . The meta for the level may be added to the output as well and looking to hear on suggestions @Edouard-Legoupil.

UGA-ITOSHumanitarianGIS commented 1 year ago

@fititnt and @Edouard-Legoupil We have added the cache to the Mexico P-Code service as it was not returning geojson exports. The geojson results are now available using the similar scheme as mentioned earlier in this thread: https://apps.itos.uga.edu/codv2api/api/v1/themes/cod-ab/locations/MEX/versions/current/geoJSON/1

which downloads the geojson with results to use in your PowerBI, QGIS, webapps or other solutions. We will test it in R and if you have the opportunity to provide feedback...it returns a file instead of a stream.

Regarding the services in general: Test results on non cache old server yield 4.85 seconds to return the results to the updated service in the new environment 2.22 seconds. The update for Mexico is a workaround as we've implemented the ogr2ogr for exports to a file which then is uploaded for access through the api, so a little different from other countries for now