XML scraper?

[cpsievert]

I work with XML more than I care to admit and I'm always looking for excuses to build more web scrapers using my XML2R package. Does anyone have an XML database in mind?

[sckott]

Cool! A few thoughts:

• In the solr package I'm working on you can ask for json or xml. The search functions give back json or xml or parsed data to lists or data.frames. The JSON parsing is easier, but I haven't got around to solving all the xml parsing. There are many different ways data may come back depending on the users search. For example, the xml parsing isn't done at all here https://github.com/ropensci/solr/blob/master/R/parsers.r#L508-L514
• @cboettig has been working on RNeXML https://github.com/ropensci/RNeXML - an R interface for working with nexml, a specification for phylogenetic data and metadata (see also this paper) - not sure if he needs any help?
• I think we are working on building out a new API for Fishbase data. We currently just parse xml files stored locally. Not sure if the new API will give back XML or JSON. maybe @cboettig has more infos?

[karthik]

This would be great, @cpsievert. I've put off working on the aRxiv package (an interface to arxiv) mostly because of the nastiness of dealing with XML. Not sure if this repo is on GitHub but if not I'll push it shortly.

[karthik]

@sckott The new fishbase API does json so no need to go back to xml there.