Closed jenmark closed 9 years ago
hi @jenmark - dismo
uses an old API that GBIF no longer supports - if you go to data.gbif.org
you see
data.gbif.org has now been decommissioned. Please use www.gbif.org for all data access
The spocc
package uses rgbif
to get at GBIF data. rgbif
uses the new GBIF APIs. You can get GBIF data from spocc
or rgbif
. In spocc
, see the function occ()
, and in rgbif
, you most likely want occ_search()
, but you may want to use rgbif
to look up each name first, using name_backbone()
to get an ID for each name, then pass that ID to occ_search
, like
key <- name_backbone(name='Helianthus annuus', kingdom='plants')$speciesKey
occ_search(taxonKey=key, limit=20)
#> returns data...
does that help?
Hi @sckott ,
Thank you very much, that explains things! I'll use rgbif instead - thanks for the tutorials link and examples.
Much appreciated, Jennifer
hi again, saw a few more questions in your email:
About invalid integer range
- There was a similar question recently - here was that discussion https://github.com/ropensci/rgbif/issues/143 Basically, when you get the result from name_backbone()
you may not have all taxonomic keys in that list - you may have some NULL
values or other things. Check that list to make sure. You can also simply search by name in occ_search()
- see first example in https://github.com/ropensci/rgbif#search-for-occurrence-data
I don't know of any way to make the selection random. I imagine you want a random selection within each species? I guess if I wanted 50 random records, I might get e.g., 1000 occurrence record keys for each species, then select 50 at random from that list of 1000, then retrieve those records. e.g.
out <- occ_search(scientificName = 'Ursus americanus', fields="key", limit=1000)
keys <- sample(out$data$key, 50) # get random set of 50
occ_get(keys, fields = "all") # pass those keys to occ_get, set fields to all to get all fields
is relatively quick.
Hi Scott,
Thank you, that's a big help - retrieving 1000 and then randomly selecting 50 of them (for example) is a good solution! I'll try it.
A colleague has just helped me get the records without the "invalid request" error, but thanks for explaining further - it's useful to know what was going wrong.
I hope you don't mind one final quick question - I'm trying to save the output results in csv file, and found an example here: http://stackoverflow.com/questions/27089159/how-do-i-save-individual-species-data-downloaded-via-rgbif
But that is for a list and not the full output as below.
Please could you advise me how to save the output? I need the columns: name; key; decimal.latitude; decimal.longitude; locality; countryCode; country; institutionCode.
Thanks in advance - any advise is really appreciated!
?
What we have done is:
minitest<-read.csv("C:/Users/Jenny/Documents/R/minitest_rgbif.csv",stringsAsFactors=FALSE) head(minitest) minitest2<-as.vector(minitest$Accepted.Binomial) keys2 <- sapply(minitest2, function(x) name_backbone(name=x)$speciesKey, USE.NAMES=FALSE) search_res<-occ_search(taxonKey=keys2, limit=2,hasCoordinate=TRUE)
keys3<-unlist(keys2) search_res<-occ_search(taxonKey=keys3, limit=2,hasCoordinate=TRUE) search_res
filenames <- paste(sapply(sapply(search_res, FUN = "[[", "name", simplify = FALSE), unique), ".txt", sep = "") mapply(search_res, filenames, FUN = function(x, y) write.table(x, file = y, row.names = FALSE))
Error in data.frame(meta = list(offset = 0L, limit = 2L, endOfRecords = FALSE, : arguments imply differing number of rows: 1, 7, 2, 0
str(search_res)
From: Scott Chamberlain notifications@github.com Sent: Tuesday, February 24, 2015 4:15 PM To: ropensci/spocc Cc: Jennifer Mark Subject: Re: [spocc] dismo error "invalid request" when retrieving gbif occurrence data - please help! (#113)
hi again, saw a few more questions in your email:
About invalid integer range - There was a similar question recently - here was that discussion ropensci/rgbif#143https://github.com/ropensci/rgbif/issues/143 Basically, when you get the result from name_backbone() you may not have all taxonomic keys in that list - you may have some NULL values or other things. Check that list to make sure. You can also simply search by name in occ_search() - see first example in https://github.com/ropensci/rgbif#search-for-occurrence-data
I don't know of any way to make the selection random. I imagine you want a random selection within each species? I guess if I wanted 50 random records, I might get e.g., 1000 occurrence record keys for each species, then select 50 at random from that list of 1000, then retrieve those records. e.g.
out <- occ_search(scientificName = 'Ursus americanus', fields="key", limit=1000) keys <- sample(out$data$key, 50) # get random set of 50 occ_get(keys, fields = "all") # pass those keys to occ_get, set fields to all to get all fields
is relatively quick.
Reply to this email directly or view it on GitHubhttps://github.com/ropensci/spocc/issues/113#issuecomment-75787214.
BU is a Disability Two Ticks Employer and has signed up to the Mindful Employer charter. Information about the accessibility of University buildings can be found on the BU DisabledGo webpages This email is intended only for the person to whom it is addressed and may contain confidential information. If you have received this email in error, please notify the sender and delete this email, which must not be copied, distributed or disclosed to any other person. Any views or opinions presented are solely those of the author and do not necessarily represent those of Bournemouth University or its subsidiary companies. Nor can any contract be formed on behalf of the University or its subsidiary companies via email.
hi, for fields to save, you can either request those in the occ_search()
request like
occ_search(scientificName = "Ursus americanus", limit = 5, fields=c("name", "key", "decimalLatitude", "decimalLongitude", "locality", "countryCode", "country", "institutionCode"))
#> Records found [6857]
#> Records returned [5]
#> No. unique hierarchies [1]
#> No. media records [5]
#> Args [scientificName=Ursus americanus, limit=5, offset=0,
#> fields=name,key,decimalLatitude,decimalLongitude,locality,countryCode,country,institutionCode]
#> First 10 rows of data
#>
#> name key decimalLongitude decimalLatitude countryCode country institutionCode
#> 1 Ursus americanus 891034709 -103.29468 29.23322 US United States iNaturalist
#> 2 Ursus americanus 1024328693 -118.14681 34.20990 US United States iNaturalist
#> 3 Ursus americanus 891045574 -72.52534 43.73511 US United States iNaturalist
#> 4 Ursus americanus 891041363 -103.28908 29.28284 US United States iNaturalist
#> 5 Ursus americanus 1050834838 -107.70675 33.11070 US United States MSB
#> Variables not shown: locality (chr)
for saving the data, I'm not sure exactly what's going wrong there, I would suggest trying the for loop solution in that Stackoverflow answer, it should be easier to understand what's going wrong there...
?Thanks very much, Scott
I'll try these out. Thanks for all your help!
All the best,
Jennifer
From: Scott Chamberlain notifications@github.com Sent: Tuesday, February 24, 2015 5:40 PM To: ropensci/spocc Cc: Jennifer Mark Subject: Re: [spocc] dismo error "invalid request" when retrieving gbif occurrence data - please help! (#113)
for saving the data, I'm not sure exactly what's going wrong there, I would suggest trying the for loop solution in that Stackoverflow answer, it should be easier to understand what's going wrong there...
Reply to this email directly or view it on GitHubhttps://github.com/ropensci/spocc/issues/113#issuecomment-75805079.
BU is a Disability Two Ticks Employer and has signed up to the Mindful Employer charter. Information about the accessibility of University buildings can be found on the BU DisabledGo webpages This email is intended only for the person to whom it is addressed and may contain confidential information. If you have received this email in error, please notify the sender and delete this email, which must not be copied, distributed or disclosed to any other person. Any views or opinions presented are solely those of the author and do not necessarily represent those of Bournemouth University or its subsidiary companies. Nor can any contract be formed on behalf of the University or its subsidiary companies via email.
Dear rOpenSci,
I'm using R to download occurrence records for multiple species at once (from csv) from GBIF with package dismo. I have some modified code for this, but am consistently getting the error "invalid request".
This code worked 6 months ago, but now it returns the error - I don't know if something has changed with dismo, GBIF or if I'm making an error, but I'm using the most up to date dismo version. Could it be to do with the gbif API dismo uses?
My apologies if I should have posted this elsewhere - I'm new to GitHub and rOpenSci.
Please can anyone help? Many thanks in advance, Jennifer
My code:
Read csv file of accepted Genus and species names (2 columns) - call it 'd':
d <- read.csv("Accepted_2.csv")
Read the column headings ('New.Genus' and 'New.Species') as characters, rather than names to extract from GBIF:
d$New.Genus<-as.character(d$New.Genus)
d$New.Species<-as.character(d$New.Species)
attach(d)
Try extracting the first species on the csv file list, from GBIF online database:
d1<-try(gbif(d$New.Genus [1],d$New.Species[1],geo=T,sp=F))
Returns error “invalid request”:
Loading required package: XML http://data.gbif.org/ws/rest/occurrence/count?scientificname=Alexa+imperatricis&coordinatestatus=true Error in gbif(d$New.Genus[1], d$New.Species[1], geo = T, sp = F) : invalid request
Rest of code:
Repeat this process for all species in the csv file. Write the extracted x/y co-ordinate data in new csv file, named "Accepted2_output.csv":