opensdmx / rsdmx

Tools for reading SDMX data and metadata in R
https://github.com/opensdmx/rsdmx/wiki
105 stars 30 forks source link

Beginner issues getSDMXServiceProviders #78

Closed pssguy closed 8 years ago

pssguy commented 8 years ago

Just downloaded from CRAN and using your QuickStart

library(rsdmx)

# list of providers
providers <- getSDMXServiceProviders()
as.data.frame(providers)

Error in as.data.frame.default(x[[i]], optional = TRUE) : 
  cannot coerce class "structure("SDMXServiceProvider", package = "rsdmx")" to a data.frame
sessionInfo()

R version 3.2.2 (2015-08-14)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 8 x64 (build 9200)

locale:
[1] LC_COLLATE=English_Canada.1252  LC_CTYPE=English_Canada.1252    LC_MONETARY=English_Canada.1252 LC_NUMERIC=C                   
[5] LC_TIME=English_Canada.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] rsdmx_0.5-0

loaded via a namespace (and not attached):
[1] plyr_1.8.3     tools_3.2.2    RCurl_1.95-4.7 Rcpp_0.12.2    bitops_1.0-6   XML_3.98-1.3  
eblondel commented 8 years ago

Thanks @pssguy for reporting this issue,i 've checked and indeed something is wrong. I think i will do ASAP the release the revision in preparation (0.5-1) to make the the fix available in CRAN. Actually it seems it has been already fixed in the current Github version. I will add unit test (seems missing) to properly control its functioning.

For the time being, as alternative, you can install the latest rsdmx from Github, using the devtools package:

require(devtools)
install_github("opensdmx/rsdmx")

and run again the code

providers <- getSDMXServiceProviders()
as.data.frame(providers)

Let me know if it helps

pssguy commented 8 years ago

Thanks that did the trick

Also solved problem with reading concept schemes

csUrl <- "http://data.fao.org/sdmx/registry/conceptscheme/FAO/ALL/LATEST/?detail=full&references=none&version=2.1"
 csobj <- readSDMX(csUrl)
Premature end of data in tag u line 1 etc.

This looks like a really powerful package. Is there a tutorial out there which works through the process from idea to final data.frame. For instance, say i was interested in immigration by country of Syrian nationals. I can see that the OECD example has that kind of information but am not sure how to use all your functions to get to correct portal and URL, years available etc.

eblondel commented 8 years ago

Thanks @pssguy For the timebeing there is such a tutorial available. For the OECD, one immediate approach you could follow is to access the [OECD Stats portal](http://stats.oecd.org/=, there you can access datasets, apply your filter, and then export as SDMX GetData request URL. Then copy your url and apply readSDMX on it.

In parallel, i would also like to suggest such tutorial you mention, even providing some helper functions to filter data, and possibly to develop some basic R shiny web-app to access such information and facilitate the data retrieving.

pssguy commented 8 years ago

@eblondel Thanks for reply. I think you mean no such tutorial?

Yes the method you proposed was what I had done. I plan to produce some charts based on the data and have previously looked at searching acs data so will revisit. I'll post a URL when it is done

The actual data download can take a long time unless there is a filter on countries/years which would probably be needed for a shiny app

eblondel commented 8 years ago

Yes it's true that the filtering is important, because datasets may be huge indeed. I started investigating a shiny client that may extend rsdmx and help users do their data query in an interactive way, and get back the appropriate data in R. I'm going to close this ticket related to getSDMXServiceProviders(). Please feel free to ask me further support for using the package, and report other bugs.