search

AtlasOfLivingAustralia / galah-R

Query living atlases from R
https://galah.ala.org.au
38 stars 3 forks source link

Offline downloads (called by `atlas_occurrences`) fail for some atlases #156

Closed mjwestgate closed 3 months ago

mjwestgate commented 1 year ago

Affected atlases:

Using Estonia as an example; this atlas appears to have an offline downloads API at https://elurikkus.ee/biocache-service/occurrences/offline/download. While using wait_for_download() (within atlas_occurrences), I got the following status URL:

https://elurikkus.ee/biocache-service/occurrences/offline/status/5fe609b1-370f-3f3f-8482-8e46c6d94551-1663130411098

This resolves to:

{"status":"finished","downloadUrl":"http://biocache.ala.org.au/biocache-download/5fe609b1-370f-3f3f-8482-8e46c6d94551/1663130411098/data.zip"}

Clearly this isn't the correct downloadUrl. Manually replacing http://biocache.ala.org.au/ with https://elurikkus.ee/ also fails.

For France, the status queue info returned is:

$message
[1] "Already in queue."

$status
[1] "inQueue"

$queueSize
[1] 1

$statusUrl
[1] "http://biocacheservice:8080/biocache-service/occurrences/offline/status/6a784afe-b089-30b4-ac31-6e64e82dffc5-1663201057954"

$searchUrl
[1] "http://localhost/ala-hub/occurrences/search?&q=*%3A*&fq=%28year%3A%221970%22%29+AND+%28order%3A%22Lagomorpha%22%29"

attr(,"url")
[1] "http://portail.gbif.fr/biocache-service/occurrences/offline/download?fq=%28year%3A%221970%22%29%20AND%20%28order%3A%22Lagomorpha%22%29&fields=taxon_name%2Cyear&qa=none&emailNotify=false&sourceTypeId=2004&reasonTypeId=4&email=ala4r%40ala.org.au&dwcHeaders=true"

Portugal generates a download url that appears valid, but does not resolve:

https://registos.gbif.pt/biocache-download/5fe609b1-370f-3f3f-8482-8e46c6d94551/1663202568148/data.zip

UK sends an email notifying that download has failed, but as with Portugal, I haven't worked out why/how yet.

vjrj commented 1 year ago

cc @keithraven-nbn @rpfigueira @sylmorin-gbif

vjrj commented 1 year ago

Probably the same like https://github.com/AtlasOfLivingAustralia/galah/issues/165#issuecomment-1294483112

mjwestgate commented 7 months ago

This is still an issue for Portugal and Estonia; UK and France supported as of version 2.0

mjwestgate commented 3 months ago

Gonna close this one; we can add more specific errors in due course