Check API for new features

vincentarelbundock commented 3 years ago

Links initially posted by @etiennebacher

For reference, here are a few links that could give ideas for new features:

New features in V2
Advanced Data API Queries (BETA), to get the dataset as it existed in 1997 for example.
Other World Bank APIs (this is maybe out of the scope of WDI, but it gives ideas).

vincentarelbundock commented 3 years ago

For the record, I don't have a restricted view on package scope necessarily, beyond time constraints. Everything on the WB API page should be fair game if anyone is interested in adding it.

etiennebacher commented 3 years ago

Hello, after checking the page with V2 new features, I think that it would be useful to add two of them in WDI:

observation status, e.g is the number a forecast? I think this could be added in extra, to avoid an additional argument.
download, either in csv or excel. I think this would be especially useful for those who use WDIsearch(), and want to store the dataset in their folder (more robust than downloading data each time we work on the script because the latter requires internet connection). Of course, the user could write the data to csv right after it was downloaded in the R session, but this adds some steps. A potential drawback is that it requires adding two arguments to WDI: one to know if the user wants to download and in which format (we could call this argument download, with possible values being NULL, "csv", or "excel"), and one argument to specify the path.

What do you think? I know that you want to limit the number of arguments in WDI, but this download option could be useful.

If you're okay with that, I can do the changes and prepare a PR.

vincentarelbundock commented 3 years ago

Thank for looking into this!

Adding observation status to extra sounds like a no brainer. Great!

On download, how does this compare to WDIbulk and WDIcache?

(Note: I'm on semi-vacation with sporadic access to Github, so I might not react super quickly)

etiennebacher commented 3 years ago

If I'm correct, WDIcache does not provide any data but a list of indicators, and WDIbulk downloads all WDI indicators without exception. The argument download would be used in WDI to download a csv/excel file that contains only the data we want. For instance, if I already have:

WDI("FR", "SP.POP.TOTL", start = 1990, end = 2000)

then download would only create a csv containing "SP.POP.TOTL" for France between 1990 and 2000.

I'll do a PR at least to include observation status.

vincentarelbundock commented 3 years ago

Sounds good on the PR.

Honestly, I'm not convinced by the download option. In my own work, when I need to ensure 100% replicability, I typically call WDI once and then save the results as a CSV or RDS file. Then, I comment out the WDI call and load from file.

etiennebacher commented 3 years ago

Right, I do some similar stuff too. I'll only focus on the obs status then, probably PR this weekend.

vincentarelbundock commented 3 years ago

sounds great, thanks. I do think the observation status is super cool.

vincentarelbundock commented 2 years ago

Closing this now since I don't think we had anything concrete planned. Feel free to open new issues if you have specific feature requests or ideas.

Thanks!

vincentarelbundock / WDI

Check API for new features #42