Using R to extract data from web APIs

[jamiecmontgomery]

@tclavelle has offered to lead a session on using R to get data from web APIs.

Tentatively, this will be scheduled in mid to late June. Waiting on a couple of other sessions to get sorted before putting this one on the calendar.

[jamiecmontgomery]

We've set this up for June 8th! @tclavelle if you wouldn't mind writing up a little blurb for members to give a bit of info for what to expect that would be most excellent!

[sharilaster]

If you're still interested, we'd be happy to host this session in the library's Collaboratory.

[jamiecmontgomery]

A reminder that this Thursday, June 8th, @tclavelle will be leading this session at Bren in room 1424 from 11 - 12:30. I will send out a full email + links later today or tomorrow but please keep this on your calendar!

If you are planning on attending please indicate your interest by adding a reaction (hit the +:smile: in the upper right corner of this issue)

Thank you @sharilaster for the offer! The collaboratory would be a great place to host some of our sessions. I will be in touch earlier on when we are planning our next one to see if it makes sense to host it at the library.

[jamiecmontgomery]

Here is Tyler's blog post for today's session: https://tclavelle.github.io/blog/r_and_apis/

[stapial]

Thanks Jamie and Tyler!

I missed what looked like a really interesting session. I have a final, pushing into my repo real good

https://www.youtube.com/watch?v=vCadcBR95oU

[jcvdav]

As a follow-up on today's discussion after @tclavelle great session:

Perhaps some of you know this project, but I thought I'd share just in case... There seems to be a project (OpenFisheries) going on, seeking to make FAO data available. From their About page:

Currently the site's data holdings include global capture fishing landings from the Food and Agriculture Organization (FAO) of the United Nations. In near future we hope to augment this information with spatial species distribution data and expand our collection with fish price data where available.

Running something like this will get you a familiar graph:

extract <- function(x) {
  df <- data_frame(year = x$year,
                   catch = x$catch)
}

GET(url = "http://openfisheries.org/api/landings.json") %>% 
  content() %>% 
  lapply(extract) %>%
  bind_rows() %>% 
  ggplot(aes(x = year, y = catch)) +
  geom_point() +
  geom_line() +
  theme_bw()

Seems like it is growing slowly, but might be worth to bookmark their page and / or see a way to collaborate with them (?).

[bbest]

Also bummed to have missed @tclavelle's no doubt awesome sesh today.

I like @jcvdav's example -- you can see the pros at rOpensci using the same functions in their rfisheries package, eg in the landings() function: landings.R#L31-L33:

 landings_call <- GET(url, foptions)
 stop_for_status(landings_call)
 landings_data_JSON <- content(landings_call)

There might already be an R package to fetch the type of data you want from a web API, but knowing the R functions @tclavelle references in Using R to extract data from web APIs from the httr and jsonlite R packages helps to debug any problems and enables you to extract exactly what you want, regardless of an existing R package.

[tclavelle]

Thanks guys! Great find and awesome example @jcvdav. I should bing the OpenFisheries people and ask them to include the FAO aquaculture and commodities trade data as well and thus never have to touch FishStatJ ever again.

On Thu, Jun 8, 2017 at 7:46 PM, Ben Best notifications@github.com wrote:

Also bummed to have missed @tclavelle https://github.com/tclavelle's no doubt awesome sesh today.

I like @jcvdav https://github.com/jcvdav's example -- you can see the pros at rOpensci https://ropensci.org/packages/ using the same functions in their rfisheries https://ropensci.org/tutorials/rfisheries_tutorial.html package, eg in the landings() function: landings.R#L31-L33 https://github.com/ropensci/rfisheries/blob/master/R/landings.R#L31-L33:

landings_call <- GET(url, foptions) stop_for_status(landings_call) landings_data_JSON <- content(landings_call)

There might already be an R package to fetch the type of data you want from a web API, but knowing the R functions @tclavelle https://github.com/tclavelle references in Using R to extract data from web APIs https://tclavelle.github.io/blog/r_and_apis/ from the httr and jsonlite R packages helps to debug any problems and enables you to extract exactly what you want, regardless of an existing R package.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/eco-data-science/eco-data-science.github.io/issues/36#issuecomment-307280693, or mute the thread https://github.com/notifications/unsubscribe-auth/AHyNJ4glF2j9MnbCt3I5nIpjYqxbWNu8ks5sCLH5gaJpZM4M6e_g .

-- Tyler Clavelle Project Researcher Sustainable Fisheries Group (SFG) University of California, Santa Barbara tclavelle@bren.ucsb.edu (802) 578-8701