Submission: ruODK, Client for the ODK Central API

[florianm]

Submitting Author: Florian Mayer (@florianm)
Repository: https://github.com/dbca-wa/ruODK Version submitted: 0.6.6.9025 (Jun 2020) (originally 0.6.0) Editor: @maelle
Reviewer 1: @karissawhiting
Reviewer 2: @jmt2080ad
Archive: 2020-02-11 Version accepted: TBD

---

• Paste the full DESCRIPTION file inside a code block below:

Type: Package
Package: ruODK
Title: An R Client for the ODK Central API
Version: 0.6.6.9025
Authors@R: 
    c(person(given = c("Florian", "W."),
             family = "Mayer",
             role = c("aut", "cre"),
             email = "Florian.Mayer@dbca.wa.gov.au",
             comment = c(ORCID = "0000-0003-4269-4242")),
      person(given = "Maëlle",
             family = "Salmon",
             role = "rev",
             email = "maelle.salmon@yahoo.se",
             comment = c(ORCID = "0000-0002-2815-0399")),
      person(given = "Karissa",
             family = "Whiting",
             role = "rev",
             comment = c(ORCID = "0000-0002-4683-1868")),
      person(given = "Jason",
             family = "Taylor",
             role = "rev"),
      person(given = "DBCA",
             role = c("cph", "fnd")),
      person(given = "NWSFTCP",
             role = "fnd"))
Description: Utilities to access and tidy up data from ODK
    Central's API.  ODK Central is OpenDataKit's clearinghouse for
    digitally captured data <https://docs.opendatakit.org/central-intro/>.
    ODK Central's API is documented at
    <https://odkcentral.docs.apiary.io/>.
License: GPL-3
URL: https://dbca-wa.github.io/ruODK/,
    https://github.com/dbca-wa/ruODK
BugReports: https://github.com/dbca-wa/ruODK/issues
Depends: 
    R (>= 3.4)
Imports: 
    clisymbols (>= 1.2.0),
    crayon (>= 1.3.4),
    dplyr (>= 0.8.5),
    fs (>= 1.4.1),
    glue (>= 1.4.0),
    httr (>= 1.4.1),
    janitor (>= 2.0.1),
    lifecycle (>= 0.1.0),
    lubridate (>= 1.7.8),
    magrittr (>= 1.5),
    purrr (>= 0.3.4),
    readr (>= 1.3.1),
    rlang (>= 0.4.5),
    rlist (>= 0.4.6.1),
    stringr (>= 1.4.0),
    tibble (>= 2.1.3),
    tidyr (>= 1.0.3),
    tidyselect (>= 1.0.0),
    xml2 (>= 1.2.2)
Suggests: 
    covr (>= 3.4.0),
    DT (>= 0.9),
    ggplot2 (>= 3.2.1),
    knitr (>= 1.26),
    leaflet (>= 2.0.3),
    listviewer (>= 3.0.0),
    rmarkdown (>= 1.17),
    roxygen2 (>= 7.1.0),
    testthat (>= 2.3.2),
    usethis (>= 1.6.0),
    vcr (>= 0.5.4)
VignetteBuilder: 
    knitr
RdMacros: 
    lifecycle
Encoding: UTF-8
Language: en_AU
LazyData: true
RoxygenNote: 7.1.0
X-schema.org-applicationCategory: Data Access
X-schema.org-keywords: database, open-data, opendatakit, odk, api, data, dataset

Scope

• Please indicate which category or categories from our package fit policies this package falls under: (Please check an appropriate box below. If you are unsure, we suggest you make a pre-submission inquiry.):

  • [X] data retrieval
  • [ ] data extraction
  • [ ] database access
  • [ ] data munging
  • [ ] data deposition
  • [ ] reproducibility
  • [ ] geospatial data
  • [ ] text analysis

• Explain how and why the package falls under these categories (briefly, 1-2 sentences):

Data retrieval: ruODK retrieves data from ODK Central, a data clearinghouse containing data which have been digitally captured by the data collection app ODK Collect using Xforms.

As mentioned by @annakrystalli, the below categories are touched by ruODK, but don't apply:

Data extraction or munging: ruODK transforms and sanitises the data from ODK Central from the original format (which parses to nested lists in R) into tibbles.

Reproducibility: ruODK allows to script and repeat the data extraction step - the main use case it is being written for.

Geospatial data: while ODK allows to capture location data (points, lines, polygons), and ruODK extracts these values, ruODK is not primarily a spatial package.

• Who is the target audience and what are scientific applications of this package?

(Scope and use cases are also mentioned in the README.)

Any organisation collecting data with the OpenDataKit suite will need to extract the data from the data clearinghouses, ODK Aggregate (outgoing) or ODK Central (new). Some may want to analyse the data straight out of ODK Central, some may need to transfer the data into another data warehouse for further post-processing, QA, and integration with other data sources.

For both use cases, ruODK bridges and simplifies the gap between the data sitting in ODK Central, and the data being a tibble in R, ready for further processing.

In a nutshell, ruODK aims to be to ODK Central what ckanr is to CKAN.

As per release notes for ODK Central 0.6, next to the other options, ruODK is now the recommended package to access ODK Central data in R.

• Are there other R packages that accomplish the same thing? If so, how does yours differ or meet our criteria for best-in-category?

• If you made a pre-submission enquiry, please paste the link to the corresponding issue, forum post, or other discussion, or @tag the editor you contacted.

https://github.com/ropensci/software-review/issues/328 Thanks to @annakrystalli for feedback on the pre-submission. Changes since then: added remaining functions.

Technical checks

Confirm each of the following by checking the box. This package:

• [x] does not violate the Terms of Service of any service it interacts with.
• [X] has a CRAN and OSI accepted license.
• [X] contains a README with instructions for installing the development version.
• [X] includes documentation with examples for all functions.
• [X] contains a vignette with examples of its essential functions and uses.
• [X] has a test suite.
• [X] has continuous integration, including reporting of test coverage using services such as Travis CI, Coveralls and/or CodeCov.

Publication options

Note: I would like to submit a paper about the package in a few weeks, but haven't got the manuscript ready and approved for publication just yet.

• [X] Do you intend for this package to go on CRAN? > if you see fit, yes please (0.6.6)
• [ ] Do you wish to automatically submit to the Journal of Open Source Software? If so:

JOSS Options

- [ ] The package has an **obvious research application** according to [JOSS's definition](https://joss.readthedocs.io/en/latest/submitting.html#submission-requirements). - [ ] The package contains a `paper.md` matching [JOSS's requirements](https://joss.readthedocs.io/en/latest/submitting.html#what-should-my-paper-contain) with a high-level description in the package root or in `inst/`. - [ ] The package is deposited in a long-term repository with the DOI: - (*Do not submit your package separately to JOSS*)

• [ ] Do you wish to submit an Applications Article about your package to Methods in Ecology and Evolution? If so:

MEE Options

- [ ] The package is novel and will be of interest to the broad readership of the journal. - [ ] The manuscript describing the package is no longer than 3000 words. - [ ] You intend to archive the code for the package in a long-term repository which meets the requirements of the journal (see [MEE's Policy on Publishing Code](http://besjournals.onlinelibrary.wiley.com/hub/journal/10.1111/(ISSN)2041-210X/journal-resources/policy-on-publishing-code.html)) - (*Scope: Do consider MEE's [Aims and Scope](http://besjournals.onlinelibrary.wiley.com/hub/journal/10.1111/(ISSN)2041-210X/aims-and-scope/read-full-aims-and-scope.html) for your manuscript. We make no guarantee that your manuscript will be within MEE scope.*) - (*Although not required, we strongly recommend having a full manuscript prepared when you submit here.*) - (*Please do not submit your package separately to Methods in Ecology and Evolution*)

Code of conduct

• [X] I agree to abide by rOpenSci's Code of Conduct during the review process and in maintaining my package should it be accepted.

Update 31 Aug: pasted new DESCRIPTION with version 0.6.1 (addressing all comments below). Update 14 Sept: 0.6.3 after tidyr 1.0.0 moves from dev to stable dependency Update 18 Sept: 0.6.4 uses new example data for tests and vignettes Update 23 Sept: 0.6.5 uses lifecycle badges Update 25 Sept: 0.6.6 odata_submission_get() parses data, dates, and downloads attachments Update 10-18 Oct: met ODK developers and users; met @jmt2080ad, met sckott (Scott Chamberlain), held four ruODK workshops (Perth, Seattle, Portland), got constructive feedback Update 27 Nov: 0.6.6.900* addressing reviewer comments, preparing for ODK Central 0.7 release Update 12 Feb 2020: reviewer comments addressed, some improvements added, ready for reviewer response Update 15 Jun 2020: reviewer response addressed, parsers added for geotrace and geoshape

[maelle]

Editor checks:

• [x] Fit: The package meets criteria for fit and overlap
• [x] Automated tests: Package has a testing suite and is tested via Travis-CI or another CI service.
• [x] License: The package has a CRAN or OSI accepted license
• [x] Repository: The repository link resolves correctly
• [ ] Archive (JOSS only, may be post-review): The repository DOI resolves correctly
• [ ] Version (JOSS only, may be post-review): Does the release version given match the GitHub release (v1.0.0)?

---

Editor comments

:wave: @florianm! Thanks a lot for your submission! :grin: I have a few questions/asks I'd like to see tackled before I start looking for reviewers. I am also waiting to run the local checks because I don't have credentials yet, see my forum post.

• I see a few open issues in your repo. Could you clarify here and in their description whether you're seeking advice on the functionalities implementation/usefulness? Why not solve some of them before the reviews of the package? It'd be better for the functionality to be implemented so the reviewers can review them too. It is fine if we put this issue on hold for a bit in order for you to have time to work on the enhancements if needed, and you could ask for help in rOpenSci channels.

• Reg "Note: I would like to submit a paper about the package in a few weeks, but haven't got the manuscript ready and approved for publication just yet." This is also a good reason to put the issue on hold since the reviewers would read the short JOSS paper. What do you mean by approved?

• I do not understand "Especially in these trying times, it is important to ask: “ruODK?”" and " u r ODK!" in the README, could you add footnotes or so there? Unless it is very obvious :see_no_evil: (play on words with OK? to my defence and the defence of my suggestion, my not being a native speaker might have made this less clear :joy: )

• Likewise I do not understand the use of %<% when describing use cases.

• There is a link missing " [ODK forum]". How should the future reviewers best ask for credentials if they're new to the service? I wasn't too sure about my own post.

• The current error messages are not informative in the absence of credentials, i.e. I get Error in curl::curl_fetch_memory(url, handle = handle) : <url> malformed if I run project_list() now without credentials. If there's no system variables set and no credentials passed either, the functions should fail with an informative error message.

• Do not use the Author and Maintainer fields in DESCRIPTION, they are automatically generated from Authors@R.

• In the example in the README you source .Rprofile. This should not be done, in your code in at least one function I see you use Sys.getenv("NAMEOFTHEVARIABLE") to access the credentials which is the best practice.

• The reference section of the pkgdwon website could use some grouping. (ignore the CI advice, outdated, cf https://ropensci.org/technotes/2019/06/07/ropensci-docs/). I see you used "usethis goodies", 1) CONTRIBUTING.md still mentions the tidyverse team at least once, 2) For info here is our guidance for accepted packages.

• There's info about contributing both in CONTRIBUTING.md and in the README (release steps), could you consolidate those?

The following items is rather a suggestion.

• Is there any diagram of ODK services/a workflow with ODK that you could use/adapt to explain visually where your R package fits, as a complement of the written part of the README?

---

Reviewers: Due date:

[florianm]

Very good points, thanks! I'll handle them as issues at https://github.com/dbca-wa/ruODK/milestone/3 apologies in advance for @ bombing you, @maelle

[maelle]

No problem, I was glad to see the issues. However we strive to keep the discussion in this submission thread (similar to what happens in journals). This does not prevent opening issues (which I personally would also do for project management), but answers should also be here. So when all items are settled it'd be nice if you can write a comment here with quotes of the items and your answers. 🙂

I will answer about publication processes in the next few days.

Can you please explain to me how to finish my credentials setup to run the tests? Thanks.

[florianm]

No probs, when done I'll go through comments here and address how my issues address them. Give me a day or two to address the larger issues!

Test credentials as promised: https://github.com/dbca-wa/ruODK/blob/master/CONTRIBUTING.md#test https://dbca-wa.github.io/ruODK/CONTRIBUTING.html#test

[maelle]

Thanks! I've just realized you recommend using Rprofile but it is code run at each session opening, environment variables should be in .Renviron, I'm editing mine via usethis::edit_r_environ() right now.

[maelle]

cf https://whattheyforgot.org/r-startup.html (maybe useful to link from docs?)

[maelle]

I can run tests now but not code such as project_list() (e.g. knitting api.Rmd fails), how do I do that, more setup steps I guess? Can I use the same credentials again, changing their names?

[florianm]

Aw crap. The vignette might need settings for our internal ODK Central server. Will send settings in ca 3h, afk just now

[florianm]

You can always use explicit url, un, pw as per vignette Setup:

project_list(url=Sys.getenv("ODKC_TEST_URL"), un=..., pw=...)

[maelle]

Does the vignette use a real project or is it a sandbox project too? I'm a bit lost. :-)

[florianm]

The test settings (sandbox) are real forms with test data. The tests are read-only and non destructive, so no worries there.

The server used in the vignettes uses real forms with (so far) only a few test records. Both servers run ODK Central v0.6.

[maelle]

And you don't want random contributors to have access to the real forms from the vignette, correct? (I might be missing something obvious) A discuss.ropensci.org forum post about the general issue (using interesting examples in the vignette, without giving access to contributors, and with R CMD check still passing) might make sense?

[florianm]

Not a problem at all to scale actually! Currently I'm using the few forms/records I have so far. Once significant production data comes in, I might create a separate testing project and give both selected/invited web users (ruODK un/pw) and app users (to submit data via ODL Collect) access to just that. I think I'll get away with a manual invitation process for prospective contributors at least for the time being. How does that sound?

Also, invite to my server coming in an hour!

[maelle]

Cool so I'll then just have to add ODKC_URL / other variables to my .Renviron? Thank you!

[florianm]

Cool so I'll then just have to add ODKC_URL / other variables to my .Renviron? Thank you!

This should make way more sense now: https://dbca-wa.github.io/ruODK/CONTRIBUTING.html#test Also check your email for working settings!

[maelle]

Awesome, thank you! Got the email, will devtools::check() soon. I will be ODK now 😉

I think the instructions should not contain Sys.getenv in order to copy paste more easily to .Renviron. 🙂

[florianm]

Glad to see that urODK now :-D

I think the instructions should not contain Sys.getenv in order to copy paste more easily to .Renviron.

Happy to adjust, which instructions do you mean? Sorry, brain frazzled now.

Re authentication and settings, should I add abstraction levels like e.g. ckanr does?

• get_odkc_{test_}{url, un, pw}() to wrap Sys.getenv("ODKC_{TEST_}{URL, UN,PW}") as standard args for functions, e.g. project_list(url = get_odkc_url(), un = get_odkc_un(), ...)
• ruODK_setup(url="xxx", un="yyy", pw="zzz", test_url="...", test_un="...", test_pw="...") to wrap Sys.setenv(ODKC_{TEST_}{URL, UN, PW}) That would be an opportunity to make ruODK yell loudly but clearly about missing credentials. Let's do this. https://github.com/dbca-wa/ruODK/issues/16

[maelle]

I mean in https://dbca-wa.github.io/ruODK/CONTRIBUTING.html#test

ODKC_TEST_URL="https://sandbox.central.opendatakit.org"
ODKC_TEST_PID=14
ODKC_TEST_FID="build_Flora-Quadrat-0-2_1558575936"
ODKC_TEST_UN="your@email.com"
ODKC_TEST_PW="..."
ODKC_URL="https://odkcentral.dbca.wa.gov.au"
ODKC_UN="your@email.com"
ODKC_PW="..."

that are the lines for .Renviron.

reg your 2d question better to ping ckanr's maintainer who's an editor too :-)

[florianm]

Ah, thanks! Fixed and pushed now.

I'll work on the remaining issues https://github.com/dbca-wa/ruODK/issues/16 and https://github.com/dbca-wa/ruODK/issues/19 tomorrow. ckanr_settings look oddly familiar :-D

[maelle]

Oops I had missed your involvement there, sorry! 😁

[maelle]

Nothing to report from automatic tests, apart from this goodpractice's compliment

♥ Hurrah! Mathematical package! Keep up the delightful work!

so as soon as you have tackled the remaining issues and written answers here, and as I answer regarding MEE&rOpenSci publication process, we can move forward. :-)

[florianm]

I think I've addressed all issues now, will write up summary here as soon as I get a moment.

Last question, is there a best practice on importing packages? in ruODK I managed to keep goodpractice() and check() happy by:

• DESCRIPTION imports all packages I use in code
• DESCRIPTION suggests extra packages used in vignettes
• My code prefixes imported functions e.g. dplyr::mutate_at
• My Roxygen docs @importFrom imported functions e.g. @importFrom dplyr mutate_at

https://kbroman.org/pkg_primer/pages/depends.html reads like I can just drop all @importFrom - should I do that?

[maelle]

You could drop the @importFrom https://r-pkgs.org/namespace.html#imports but you don't have to.🙂

[florianm]

Update: issues tackled, write-up coming tomorrow.

[maelle]

Awesome, looking forward to it!

[florianm]

Hi @maelle, thanks for the editor comments and further questions. I've included all talking points in the thread, so this answer should be comprehensively addressing all issues, and as mentioned, previous messages can be hidden for clarity.

Editor comments

I see a few open issues in your repo. Could you clarify here and in their description whether you're seeking advice on the functionalities implementation/usefulness? Why not solve some of them before the reviews of the package? It'd be better for the functionality to be implemented so the reviewers can review them too. It is fine if we put this issue on hold for a bit in order for you to have time to work on the enhancements if needed, and you could ask for help in rOpenSci channels.

• Issues grouped into milestones ROpenSci (all addressed apart from publication) and ODK Central Release 1.0 which is a collection of ideas rather than bugs or missing features.

Reg "Note: I would like to submit a paper about the package in a few weeks, but haven't got the manuscript ready and approved for publication just yet." This is also a good reason to put the issue on hold since the reviewers would read the short JOSS paper. What do you mean by approved?

• https://github.com/dbca-wa/ruODK/issues/8
• JOSS paper is included here as the short form of README and vignettes. I went with the length and scope of other JOSS papers but my JOSS manuscript feels a bit thin.
• Reviewers will probably get the best picture from the README and the vignettes.
• I would like to write a paper for MEE but have no bandwidth to do so before December.
• Writing a paper for JOSS might be quicker but unfortunately excludes the MEE option.
• I would like ruODK to be part of ROpenSci - if possible - before the ODK convening in Seattle Oct 11-13 and my presentation of ODK and ruODK to RUser PDX Oct 15. (ping @sckott)

I do not understand "Especially in these trying times, it is important to ask: “ruODK?”" and " u r ODK!" in the README, could you add footnotes or so there? Unless it is very obvious see_no_evil (play on words with OK? to my defence and the defence of my suggestion, my not being a native speaker might have made this less clear joy )

• Fulfills minimum requirement of at least one bad programmer-dad-joke in package name (@jennybc https://twitter.com/JennyBryan/status/1169365834239631360 )
• Emphasises that we care (cf. ruOK?) about R users of ODK
• Clarified a bit more in README
• ruODK comes out first on Google, second on duckduckgo, first on dogpile, same on gibiru, fourth on bing.

Likewise I do not understand the use of %<% when describing use cases.

• Dropped the tongue-in cheek "and then" magrittr pipe connector between stages of workflow in README in favour of numbering

There is a link missing " [ODK forum]". How should the future reviewers best ask for credentials if they're new to the service? I wasn't too sure about my own post.

• Added more links and background to README. Thanks for the "non-ODK user" perspective!
• Found and fixed the broken ODK Forum link (14 Sep)

The current error messages are not informative in the absence of credentials, i.e. I get Error in curl::curl_fetch_memory(url, handle = handle) : <url> malformed if I run project_list() now without credentials. If there's no system variables set and no credentials passed either, the functions should fail with an informative error message.

• Really good point! Addressed in https://github.com/dbca-wa/ruODK/issues/16 - added loud and clear error messages.
• Happy to take feedback on wording.
• https://dbca-wa.github.io/ruODK/reference/yell_if_error.html
• https://dbca-wa.github.io/ruODK/reference/yell_if_missing.html
• See also complete refactor of settings (same issue), based on best practice from ckanr

Do not use the Author and Maintainer fields in DESCRIPTION, they are automatically generated from Authors@R.

• Done https://github.com/dbca-wa/ruODK/issues/17

In the example in the README you source .Rprofile. This should not be done, in your code in at least one function I see you use Sys.getenv("NAMEOFTHEVARIABLE") to access the credentials which is the best practice.

• Thanks, learned something new here! .Renviron instead of .Rprofile
• See also complete refactor of setup (issue 16)
• Refactored vignette setup, README, other vignettes, every function, every example, every test, contributing.md to use kwargs (pid = get_default_pid(), ...) set by ru_setup. Took a while, apologies!

The reference section of the pkgdwon website could use some grouping. (ignore the CI advice, outdated, cf https://ropensci.org/technotes/2019/06/07/ropensci-docs/). I see you used "usethis goodies", 1) CONTRIBUTING.md still mentions the tidyverse team at least once, 2) For info here is our guidance for accepted packages.

• Cleaned out the usethis boilerplate
• Figured out function grouping
• https://github.com/dbca-wa/ruODK/issues/18

There's info about contributing both in CONTRIBUTING.md and in the README (release steps), could you consolidate those? Can you please explain to me how to finish my credentials setup to run the tests? Thanks.

• Rewrote and consolidated contributor's in https://github.com/dbca-wa/ruODK/issues/20
• Added non-technical contributor instructions to CONTRIBUTING.md

The following items is rather a suggestion.

Is there any diagram of ODK services/a workflow with ODK that you could use/adapt to explain visually where your R package fits, as a complement of the written part of the README?

• There is now! Thanks for the suggestion, makes the overview much more palatable. Also included in JOSS paper. Mildly increases installed package size. https://github.com/dbca-wa/ruODK/issues/19

Editor's further requests in thread

You could drop the @importFrom https://r-pkgs.org/namespace.html#imports but you don't have to.

• Dropped @importFrom in favour of properly namespacing imported functions.

And you don't want random contributors to have access to the real forms from the vignette, correct? (I might be missing something obvious) A discuss.ropensci.org forum post about the general issue (using interesting examples in the vignette, without giving access to contributors, and with R CMD check still passing) might make sense?

• Added an issue template for an account request. I (and all other ODK Central Sandbox users) can create new web user accounts to be used for testing ruODK. I will make sure to stay in line with ODK's acceptable use of the sandbox.

Other improvements

• See changelog
• Use janitor to clean names
• Add convenience helpers to link attachments (by partial match of column name) and parse dates.
• Refactored OData attachment download to share same attachments between OData, CSV export, and REST methods. As we demonstrate all three methods, this leads to a smaller installed package size.
• All tests and vignettes use our public test forms on the ODK Central Sandbox. They contain only dummy data but are (earlier versions of) forms we use in anger (or will do soon after some further improvements).
• All tests run (against our own data) in the ODK Central Sandbox, which is kept up to date with the latest master branch. This way we make sure ruODK tests the latest & greatest of the real deal. Once ODK Central solidifies around v1.0, we can probably turn to using webmockr & vcr for caching test results. @sckott might have a qualified opinion on this. Outcome of discussion: not critical for now, but on backlog for 1.0 https://github.com/dbca-wa/ruODK/issues/21
• Tests require only five env vars in .Renviron (or CI settings) as per testing instructions
• Added software review and comparison to README as requested. https://github.com/dbca-wa/ruODK/issues/28 https://github.com/dbca-wa/ruODK/issues/25
• Tidyr turned 1.0.0 (happy release day, tidyr!) - moved dependency from dev 0.8.3.9000 to stable 1.0.0. https://github.com/dbca-wa/ruODK/issues/27
• Discovered binder, created https://github.com/dbca-wa/urODK as sing-along ruODK workshop, linked from ruODK README

Output from goodpractice:

(no issues or suggestions)

• Previous note on package size addressed by using new example data with smallest possible photo attachments. (v0.6.4)

I think that's it! New things I've learned:

• .Renviron for env vars
• Grouping functions by @family in pkgdown using has_concept()
• Drop usethis boilerplate Author, Maintainer, and only use Authors@R in Description
• Drop @importFrom and only use DESCRIPTION Imports/Suggests etc and proper::namespacing()
• Clearer idea of setting up secrets for testing (see contributing - test)
• Using {binder} to launch a hosted RStudio is fantastic for workshops.

[maelle]

Thanks a lot @florianm! I'll look for reviewers now. Note that once reviewers are recruited, we give them 3 weeks to complete their review, then there's time for your changes, then for their final response (and a continuing discussion) so I think your package will not be approved before the mentioned meetings, however you already get to display an rOpenSci "in review" badge, orange for now:

[![](https://badges.ropensci.org/<issue_id>_status.svg)](https://github.com/ropensci/software-review/issues/<issue_id>)

Edited to add: the badge is still grey but it'll become orange soon, the badges server will detect the status change.

I'll hide part of our discussion to make the issue thread easier to catch up with for reviewers.

[maelle]

@florianm A further request actually, sorry, I had not googled "Open Data Kit R" before now because of the existence of the pre-submission inquiry. Doing so I found odkr, could you please compare the functionalities of your and that package? Feel free to also include the other packages that are somehow related to ODK. An ideal comparison would include prose and a table. Example of comparison tables for rtweet and CoordinateCleaner. I'd like to know how ruODK is "best in class", and how the other packages either overlap or complement its functionality. Thank you!

[annakrystalli]

Sorry about this mixup both. This was totally my omission. Should have done a better job at the presubmission stage.

[maelle]

Nope, my omission too!

[florianm]

@florianm A further request actually, sorry, I had not googled "Open Data Kit R" before now because of the existence of the pre-submission inquiry. Doing so I found odkr, could you please compare the functionalities of your and that package? Feel free to also include the other packages that are somehow related to ODK. An ideal comparison would include prose and a table. Example of comparison tables for rtweet and CoordinateCleaner. I'd like to know how ruODK is "best in class", and how the other packages either overlap or complement its functionality. Thank you!

Fixed :-) (update: some more wrangling of md tables) No trouble at all, actually a great time to search and review again - two more packages (cbs) joined the fun since I last googled. Comparison added to bottom of README. TLDR: ruODK is the only R package targeted specifically at ODK Central, without any other heavy-weight dependencies, the only one (need to re-check) that works with ODK Central's OData dialect, and the only one with worked end-to-end examples. While I give the other OData packages another whirl, which other criteria could I add to the review?

[maelle]

Wow awesome work, thank you! I can't think of other criteria right now. I'll now look for reviewers!

[sckott]

Once ODK Central solidifies around v1.0, we can probably turn to using webmockr & vcr for caching test results.

i've written a section in the http testing book trying to help folks think through when to use webmockr vs. vcr vs. fakes vs. real requests https://ropenscilabs.github.io/http-testing-book/testing-considerations.html

[florianm]

@sckott cheers, I'll read those again to figure out whether/how I can prevent testing against possibly outdated API responses in case ODK Central changes.

[florianm]

In addition to intermittent server side issues, your tests may be performing queries with cached (vcr) or mocked (webmockr) responses that are no longer valid with the current state of the remote service. It may be harmless, for example, the response to some query now returns no data because the data for that entity was removed. But it could be more serious in that the remove service changed their API such that an API route is no longer available or the route name has changed, or similar.

The ODK Central API will change, and could bring breaking changes, until ca release 1.0. Therefore I want tests to run against the real deal (the test server is ODK's own public sandbox, which is always latest master), so that ruODK has to keep up with ODK Central.

Testing real HTTP interactions should be the slowest option, but has the benefit of not adding any (permanent) files to your package. Mocking tests can be very lite weight, though you can include very heavy responses.

The one thing making my tests slow is repeated download of a ca 50MB zip file, both for testing the insides (which requires some form complexity as discussed above), which could change, and testing handling repeated dl/skip logic. The above reads as that response cached in vcr would be added to the installed package which will probably make CMD check very unhappy. The best approach for ruODK might be to add a second test project with a smaller export size (maybe a few records and only one (small) file attachment) for those scenarios purely to speed up tests. ETA for that is early next week. Hope this does not hold up the submission process.

[sckott]

Therefore I want tests to run against the real deal

makes sense

which will probably make CMD check very unhappy.

I don't think the large size of the cassettes (cached http responses) affects r check, e.g., the cassettes directory in this package https://github.com/ropensci/rgbif/ is 32 MB, but running check on it throws no notes/warnings/etc - But i could be wrong -

if size of cassettes is a concern, one option is to skip tests on CRAN for which cassettes are too large to include in the pkg, and those cassette files added to .Rbuildignore

[maelle]

First reviewer recruited, thanks a lot @karissawhiting! :smiley_cat: Reviewer guide.

@florianm no big changes planned apart from this improvement of tests, correct?

[florianm]

@maelle nothing breaking! Two minor ideas:

• Add Rmd templates with default workflow steps for a running start / turorial
• Add link to a tutorial using Pentaho Kettle (different user audience, so no threat to ruODK's best in class ;-) ), see https://forum.opendatakit.org/t/automating-data-delivery-using-the-odata-endpoint-in-odk-central/22010/7

[maelle]

Ok, but master will be stabke/unchanged for the reviews? 🙂

[florianm]

Absolutely! The Rmd templates would be nice starting points e.g. for a workshop. Just pushed an update, have details to my main reply above. Update 16 Sep: added one Rmd template, see last paragraph vignette "odata". No changes to code.

@karissawhiting thanks for reviewing ruODK! If you want to run tests and build vignettes, you'll need an account at the public ODK Central sandbox. If you DM me your preferred email for that purpose on the ROpenSci Slack, I'd be happy to create an account for you on the ODK Central sandbox.

[florianm]

@maelle @karissawhiting also soliciting feedback from the ODK community in the ODK Forum here.

[florianm]

Quick note on failing check: My vignette "api" fails to build on 2 out of 3 envs. Methinks the file attachments on ODK Central's sandbox might have suffered damage. This is a good moment to switch over to a similar but newer example form, and populate that form with submissions with smaller file attachments (set the data collection device camera to take 640px photos).

TLDR https://github.com/dbca-wa/ruODK/issues/23 will likely fix tests, ETA later today if things go to plan.

Update 18 Sep: https://github.com/dbca-wa/ruODK/issues/23 did indeed fix tests. v0.6.4 has all new example data, which fixes tests (spurious errors from older example data), reduces package size (removes CMD check note on package size). @maelle you'll need a new ODKC_TEST_FID="build_Flora-Quadrat-0-4_1564384341". @karissawhiting when you're ready to run tests and build vignettes, get in touch to receive test env vars and an account on the ODK Central sandbox.

[maelle]

Second reviewer recruited, thanks @jmt2080ad! :tada: Reviewer guide

@florianm can you point both reviewers to instructions for testing to keep things smooth? Thanks!

[maelle]

Reviews are due on 2019-10-10 :smile_cat:

[florianm]

@maelle thanks heaps! @karissawhiting @jmt2080ad I've DM'd you both on the ROpenSci Slack re test credentials.

[florianm]

@karissawhiting @jmt2080ad @maelle Update, have added lifecycle badges to all functions. The resulting NOTE is a bug upstream of the lifecycle package, see https://github.com/r-lib/lifecycle/issues/22. I've added an explanation to my cran-comments.md.

Jason, feel free to contact me for test credentials on the ROpenSci slack when you're ready to commence the review. Your involvement is very appreciated!

[florianm]

Update, just pushed a minor clean-up following advice from Jenny B on the ROpenSci slack. Removed some now obsolete helpers (map_chr_hack and friends), rebuilt docs.

[florianm]

@karissawhiting @jmt2080ad @maelle apologies for the impending code churn - I found a way to make ruODK way easier to drive: odata_submission_parse() now automatically parses the raw submission data into tibbles, parses dates, and downloads attachments. This will make the "OData" vignette a bit more concise and address https://github.com/dbca-wa/ruODK/issues/6

update: version 0.6.6 simplifies the main function odata_submission_get()

[maelle]

@jmt2080ad @karissawhiting 👋 Friendly reminder that your reviews are due tomorrow on 2019-10-10. 🙂

[karissawhiting]

Package Review

Please check off boxes as applicable, and elaborate in comments below. Your review is not limited to these topics, as described in the reviewer guide

• [x] As the reviewer I confirm that there are no conflicts of interest for me to review this work (If you are unsure whether you are in conflict, please speak to your editor before starting your review).

Documentation

The package includes all the following forms of documentation:

• [x] A statement of need clearly stating problems the software is designed to solve and its target audience in README
• [x] Installation instructions: for the development version of package and any non-standard dependencies in README
• [x] Vignette(s) demonstrating major functionality that runs successfully locally – I'm not able to build vignettes without first setting up ODK credentials
• [x] Function Documentation: for all exported functions in R help
• [x] Examples for all exported functions in R Help that run successfully locally – _not all exported functions have examples, but some internal helper functions are exported that could potentially be removed from exports (e.g. prepend_uuid())_
• [x] Community guidelines including contribution guidelines in the README or CONTRIBUTING, and DESCRIPTION with URL, BugReports and Maintainer (which may be autogenerated via Authors@R).

Functionality

• [x] Installation: Installation succeeds as documented. – _remotes::install_github() works by default, but fails when build_vignettes = TRUE if you do not have your credentials stored prior to build._
• [x] Functionality: Any functional claims of the software been confirmed.
• [x] Performance: Any performance claims of the software been confirmed.
• [x] Automated tests: Unit tests cover essential functions of the package and a reasonable range of inputs and conditions. All tests pass on the local machine.
• [x] Packaging guidelines: The package conforms to the rOpenSci packaging guidelines

Final approval (post-review)

• [x] The author has responded to my review and made changes to my satisfaction. I recommend approving this package.

Estimated hours spent reviewing: 4

• [x] Should the author(s) deem it appropriate, I agree to be acknowledged as a package reviewer ("rev" role) in the package DESCRIPTION file.

---

Review Comments

ruODK provides a much-needed link between the ODK data collection/management software suite and R. The package is modular and well-structured, and the code is tidy, styled and readable. Documentation and vignettes are informative and make the project accessible through concise descriptions and links to resources as needed.

For context, this review is coming from the perspective of someone who has experience with the ODK software suite, but little experience with R API wrapper packages.

Build

• I wasn't able to build without first specifying ODK credentials (because of api.Rmd and odata.Rmd vignettes). I’m not sure if there are established best practices with API packages and this issue, but @noamross pointed out this rcites package vignette as an example of one way this is addressed in API packages. Vignettes decide whether or not to eval code chunks based on an environment variable (in this case NOT_CRAN) and both ways are tested in the Travis-CI testing grid.

README

• The diagram is extremely helpful!
• You could add a bit more description about ODK Central Sandbox in this doc to help with multiple points of entry. New prospective users may be looking for the quickest way to test and play around with ODK/ruODK to see if it fits the needs of their project, and it seems this would be via a Sandbox account. To this point, you could adjust the "ODK Central Setup" section to say something like “Create a web user account on an ODK Central instance, or request an account on Sandbox..."
• Put “(ruODK)” in backticks for consistency (in CONTRIBUTING doc as well)
• Typo in link “Publish the formto ODK Central”.

Tests and CONTRIBUTING

• All tests passed, and overall good test coverage
• There is a barrier to entry in contributing to the package and running tests because of the need for testing credentials. That being said, the CONTRIBUTING document and the new github form you set up to request credentials definitely does help with this. Is there a way to set up a public set of ODK credentials using a dummy account? If not, consider me a +1 vote for setting up cached responses in the future when the ODK API is more stable if it will provide a way for tests to work out of the box :)
• Title of CONTRIBUTING is NA.
• You still have some leftover language from the usethis contributing doc. For example: “Before you do a pull request, you should always file an issue and make sure someone from the tidyverse team…”
• "To run tests, you’ll need access to the ODK Central sandbox instance…" I think this should go right at the top or even in the “prerequisites” section of technical contributions.

NAMESPACE

• Not all exported functions have examples, but some internal helper functions are exported that may not need to be (e.g. prepend_uuid(), yell_if_missing()).
• Do you need utils-tidy-eval.R, and these rlang functions exported?

Vignettes

• I’m having some trouble distinguishing between the OData API and RESTful API functionalities. Do these workflows mirror each other? Does one workflow have functionality available that the other does not? I think some additional delineation between these could be helpful. Here are a few ideas on how to address this (suggestions only):
  • If these are parallel workflows, consider having the vignette structures mirror each other, and add the same blurb at the top of each (like “three ways to happiness") with links to the other vignette. If there are any large benefits or drawbacks of one method or the other, call them out there or in the README.
  • Does your preferred workflow consist of a mixture of both OData and RESTful calls? If so, consider creating just a typical workflow vignette that goes through a case study using both kinds of functions.
• Consider renaming vignette file names to odata-api, and restful-api

Notes on Code

• I see several functions have . <- NULL. I’m assuming this is to avoid “no visible bindings for global variables”. One solutions is to add utils::globalVariables(".") in ruODK.R instead of specifying this in each function.
• Do you need yell_if_missing() in functions that have the get functions as their default args? In those cases, it looks like if you are missing credentials the get_default_<things>() functions already do the yelling, but I may be missing some edge case here.
• ru_setup()
  • I was expecting a confirmation message of some sort. Maybe it could print ru_settings() when you run it by default?
  • Is there a way to actually check the authentication of your username/project/form credentials here or in another function?
  • ggmaps::register_google() has an optional argument to write = TRUE where it will write credentials to your .Renviron. Just wanted to point this out as a cool enhancement option.
• @examples sections are formatted as R code so you can take out \code{ } tags (e.g. ru_setup())
• odata_submission_get()
  • It could be helpful to separate arguments for parsing the data and getting attachments in this function.
  • It looks like odata_submission_get() returns all submissions, but the equivalent function for the RESTful workflow submission_get() returns one submission at a time. I expected these functions to mirror each other in terms of outputs. Perhaps it would make sense to turn this code chunk you have in the vignette into a function? This would mirror the attachment_get()/ get_one_attachment() functions too.

submissions <- tibble::tibble(
  pid = ruODK::get_test_pid(),
  fid = ruODK::get_test_fid(),
  iid = sl$instance_id,  # this is a vector of multiple instance_ids
  url = ruODK::get_test_url(),
  un = ruODK::get_test_un(),
  pw = ruODK::get_test_pw()
) %>% 
  purrr::pmap(ruODK::submission_get)

Thank you for sharing this great package!!