Wrong url on gbOpen response

[dieghernan]

Hi:

On working with the API endpoint https://www.geoboundaries.org/api/current/gbOpen/ I found that the API response provides invalid urls.

See, as an example, the request https://www.geoboundaries.org/api/current/gbOpen/ESP/ADM0/ provides:

{
   "boundaryID":"ESP-ADM0-43220588",
   "boundaryName":"Spain",
   "boundaryISO":"ESP",
   "boundaryYearRepresented":"2017",
   "boundaryType":"ADM0",
   "boundaryCanonical":"Unknown",
   "boundarySource":"Instituto de Geografico Nacional",
   "boundaryLicense":"Creative Commons Attribution 4.0 International (CC BY 4.0)",
   "licenseDetail":"CC-BY 4.0",
   "licenseSource":"centrodedescargas.cnig.es/CentroDescargas/index.jsp#",
   "boundarySourceURL":"centrodedescargas.cnig.es/CentroDescargas/index.jsp",
   "sourceDataUpdateDate":"Fri Jan 6 15:25:05 2023",
   "buildDate":"Jan 06, 2023",
   "Continent":"Europe",
   "UNSDG-region":"Europe and Northern America",
   "UNSDG-subregion":"Southern Europe",
   "worldBankIncomeGroup":"High-income Countries",
   "admUnitCount":"1",
   "meanVertices":"667002.0",
   "minVertices":"667002",
   "maxVertices":"667002",
   "meanPerimeterLengthKM":"13154.542240986726",
   "minPerimeterLengthKM":"13154.542240986726",
   "maxPerimeterLengthKM":"13154.542240986726",
   "meanAreaSqKM":"506009.4752517802",
   "minAreaSqKM":"506009.4752517802",
   "maxAreaSqKM":"506009.4752517802",
   "staticDownloadLink":"https://github.com/wmgeolab/geoBoundaries/raw/cf0a55e21db26ca0e5f734ad3222e995ecdfc1e5/releaseData/gbOpen/ESP/ADM0/geoBoundaries-ESP-ADM0-all.zip",
   "gjDownloadURL":"https://github.com/wmgeolab/geoBoundaries/raw/2fd0fc63e2e1c98e768c4a4169e31ba024c6ce66/releaseData/gbOpen/ESP/ADM0/geoBoundaries-ESP-ADM0.geojson",
   "tjDownloadURL":"https://github.com/wmgeolab/geoBoundaries/raw/2fd0fc63e2e1c98e768c4a4169e31ba024c6ce66/releaseData/gbOpen/ESP/ADM0/geoBoundaries-ESP-ADM0.topojson",
   "imagePreview":"https://github.com/wmgeolab/geoBoundaries/raw/2fd0fc63e2e1c98e768c4a4169e31ba024c6ce66/releaseData/gbOpen/ESP/ADM0/geoBoundaries-ESP-ADM0-PREVIEW.png",
   "simplifiedGeometryGeoJSON":"https://github.com/wmgeolab/geoBoundaries/raw/2fd0fc63e2e1c98e768c4a4169e31ba024c6ce66/releaseData/gbOpen/ESP/ADM0/geoBoundaries-ESP-ADM0_simplified.geojson"
}

But when trying to access gjDownloadURL: https://github.com/wmgeolab/geoBoundaries/raw/2fd0fc63e2e1c98e768c4a4169e31ba024c6ce66/releaseData/gbOpen/ESP/ADM0/geoBoundaries-ESP-ADM0.geojson I get a 404 response.

I guess the hash commit 2fd0fc63e2e1c98e768c4a4169e31ba024c6ce66 is wrong, noneless pointing that to the commit of the latest release (#193f7f6381b022580acf99ec40b1b9ade79f5934) it works:

https://github.com/wmgeolab/geoBoundaries/raw/193f7f6381b022580acf99ec40b1b9ade79f5934/releaseData/gbOpen/ESP/ADM0/geoBoundaries-ESP-ADM0.geojson

[DanRunfola]

I am looking at this now.

We've had a number of issues with 5.0 due to some github-side limits, which I am working around ASAP.

Dan

[DanRunfola]

Just some notes as I dig into this

When I run the git rev-parse on the release files ABW-ADM0, I get a different hash than what I see in the commit on github. Local: 97ae8edd1c5c8b88fae4a6ea45a51a121d7c6c19 Remote: 6321a72f80c25e465ee2fc8c3f56a2d6885ab117

This is despite there being no delta between the local and remote. Going to dig in now and see where the remote hash is coming from.

[DanRunfola]

Alright, I found the issue, but I am not sure what is actually causing it to happen.

git log --pretty=format:'%h' -p -- ./geoBoundaries-ABW-ADM0-all.zip -n 1 shows the correct hash.

The old style I was using, git rev-parse :./geoBoundaries-ABW-ADM0.-all.zip is showing an index hash, which I am not quite certain I understand.

Going to update the code to use the log now (which should be more resiliant, but requires more parsing).

[DanRunfola]

Fix is now in place locally, regenerating API now. Should have the new API files live in 5-10 minutes or so.

[DanRunfola]

Fix hsould now be live! Let me know if there are any issues.

[dieghernan]

Hi

Thanks for your swift response, but now I got a Forbidden on the api url https://www.geoboundaries.org/api/current/gbOpen/ESP/ADM0/

[DanRunfola]

I just inadvertantly broke this, trying to ensure the earlier bug didn't occur again :)

ETA for a fix is the next 10-15 minutes or so.

[DanRunfola]

Back up. Sorry about that. After about 18 months of 100% uptime we've had a rough few days, but I think the worst is behind us now.

[dieghernan]

Thanks, now https://www.geoboundaries.org/api/current/gbOpen/ESP/ADM0/ resolves to

{
  "boundaryID": "ESP-ADM0-43220588",
  "boundaryName": "Spain",
  "boundaryISO": "ESP",
  "boundaryYearRepresented": "2017",
  "boundaryType": "ADM0",
  "boundaryCanonical": "Unknown",
  "boundarySource": "Instituto de Geografico Nacional",
  "boundaryLicense": "Creative Commons Attribution 4.0 International (CC BY 4.0)",
  "licenseDetail": "CC-BY 4.0",
  "licenseSource": "centrodedescargas.cnig.es/CentroDescargas/index.jsp#",
  "boundarySourceURL": "centrodedescargas.cnig.es/CentroDescargas/index.jsp",
  "sourceDataUpdateDate": "Fri Jan 6 15:25:05 2023",
  "buildDate": "Jan 06, 2023",
  "Continent": "Europe",
  "UNSDG-region": "Europe and Northern America",
  "UNSDG-subregion": "Southern Europe",
  "worldBankIncomeGroup": "High-income Countries",
  "admUnitCount": "1",
  "meanVertices": "667002.0",
  "minVertices": "667002",
  "maxVertices": "667002",
  "meanPerimeterLengthKM": "13154.542240986726",
  "minPerimeterLengthKM": "13154.542240986726",
  "maxPerimeterLengthKM": "13154.542240986726",
  "meanAreaSqKM": "506009.4752517802",
  "minAreaSqKM": "506009.4752517802",
  "maxAreaSqKM": "506009.4752517802",
  "staticDownloadLink": "https://github.com/wmgeolab/geoBoundaries/raw/8fa7280/releaseData/gbOpen/ESP/ADM0/geoBoundaries-ESP-ADM0-all.zip",
  "gjDownloadURL": "https://github.com/wmgeolab/geoBoundaries/raw/1222c0b/releaseData/gbOpen/ESP/ADM0/geoBoundaries-ESP-ADM0.geojson",
  "tjDownloadURL": "https://github.com/wmgeolab/geoBoundaries/raw/1222c0b/releaseData/gbOpen/ESP/ADM0/geoBoundaries-ESP-ADM0.topojson",
  "imagePreview": "https://github.com/wmgeolab/geoBoundaries/raw/1222c0b/releaseData/gbOpen/ESP/ADM0/geoBoundaries-ESP-ADM0-PREVIEW.png",
  "simplifiedGeometryGeoJSON": "https://github.com/wmgeolab/geoBoundaries/raw/1222c0b/releaseData/gbOpen/ESP/ADM0/geoBoundaries-ESP-ADM0_simplified.geojson"
}

And the geojson url resolves to https://github.com/wmgeolab/geoBoundaries/raw/1222c0b/releaseData/gbOpen/ESP/ADM0/geoBoundaries-ESP-ADM0.geojson

(commit https://github.com/wmgeolab/geoBoundaries/commit/1222c0b7c876906545b1b4ae6608accd28d38989)

[DanRunfola]

@dieghernan Yes, wonderful to see that. I did some spot checking and everythign looks to be flowing right. As you likely noticed, I also swapped to the shorter hash.

Let me know if anything comes up from here.