Closed JasperHG90 closed 6 years ago
We understand your desire to keep function names the same to avoid breaking code for current users. The exception is noted.
Thanks for switching to the rOpenSci headers. Below are the comments from goodpractice
. Please address the relevant comments as I work towards finding reviewers. You can run goodpractice::gp()
on your package anytime as you fix these issues.
── GP qualtRics ────────────────────────────────────────────────────────────────
It is good practice to
✖ write unit tests for all functions, and all package code
in general. 55% of code lines are covered by test cases.
R/getSurveyQuestions.R:52:NA
R/getSurveyQuestions.R:54:NA
R/getSurveyQuestions.R:56:NA
R/getSurveyQuestions.R:57:NA
R/getSurveyQuestions.R:58:NA
... and 262 more lines
✖ use '<-' for assignment instead of '='. '<-' is the
standard, and R users and developers are used it and it is easier
to read your code for them if you use '<-'.
R/utils.R:106:11
✖ avoid long code lines, it is bad for readability. Also,
many people prefer editor windows that are about 80 characters
wide. Try make your lines shorter than 80 characters
R/assertions.R:21:1
R/assertions.R:27:1
R/assertions.R:29:1
R/assertions.R:30:1
R/assertions.R:35:1
... and 83 more lines
✖ avoid calling setwd(), it changes the global environment.
If you need it, consider using on.exit() to restore the working
directory.
tests/testthat/test-getSurvey_reads_stored_survey.R:7:3
tests/testthat/test-registerOptions_reads_from_file.R:15:3
✖ avoid sapply(), it is not type safe. It might return a
vector, or a list, depending on the input data. Consider using
vapply() instead.
R/getSurveyQuestions.R:54:20
R/getSurveyQuestions.R:57:32
R/getSurveyQuestions.R:58:34
R/getSurveyQuestions.R:59:36
R/getSurveyQuestions.R:65:21
... and 1 more lines
✖ avoid 1:length(...), 1:nrow(...), 1:ncol(...),
1:NROW(...) and 1:NCOL(...) expressions. They are error prone and
result 1:0 if the expression on the right hand side is zero. Use
seq_len() or seq_along() instead.
R/getSurveyQuestions.R:69:23
R/utils.R:464:56
✖ fix this R CMD check NOTE: Namespace in Imports field not
imported from: ‘lubridate’ All declared Imports should be used.
✖ fix this R CMD check WARNING: LaTeX errors when creating
PDF version. This typically indicates Rd problems.
✖ fix this R CMD check ERROR: Re-running with no
redirection of stdout/stderr. Hmm ... looks like a package You may
want to clean up by 'rm -rf /tmp/RtmpTvxGKw/Rd2pdf7cb12ddc5dd3'
────────────────────────────────────────────────────────────────────────────────
Reviewer 1: @kierisi Due date: Feb 25
Reviewer 2: @trinker Due date: March 7
Hi Karthik,
Thanks for your comments.
I fixed the following gp feedback:
Partially fixed or left as is:
I've added the latest gp output below.
GP qualtRics
It is good practice to
✖ write unit tests for all functions, and all package code in general. 56% of code lines are covered by test cases.
R/getSurvey.R:127:NA
R/getSurvey.R:128:NA
R/getSurvey.R:129:NA
R/getSurvey.R:130:NA
R/getSurvey.R:131:NA
... and 258 more lines
✖ avoid long code lines, it is bad for readability. Also, many people prefer editor windows that are about 80 characters wide. Try make your lines shorter than 80 characters
inst/doc/qualtRics.R:8:1
inst/doc/qualtRics.R:11:1
inst/doc/qualtRics.R:14:1
R/assertions.R:21:1
R/assertions.R:27:1
... and 85 more lines
✖ avoid calling setwd(), it changes the global environment. If you need it, consider using on.exit() to restore the working directory.
tests/testthat/test-getSurvey_reads_stored_survey.R:8:3
tests/testthat/test-getSurvey_reads_stored_survey.R:9:11
tests/testthat/test-registerOptions_reads_from_file.R:17:3
tests/testthat/test-registerOptions_reads_from_file.R:18:11
✖ fix this R CMD check WARNING: LaTeX errors when creating PDF version. This typically indicates Rd problems. LaTeX
errors found: ! LaTeX Error: File `inconsolata.sty' not found. Type X to quit or
Just a note regarding long lines that can only be long lines like error messages, you can add # nolint
at the end of those lines https://github.com/jimhester/lintr#project-configuration this way lintr
will not see them. Not te be abused of of course 👼
Reviewed all lines and added # nolint
where appropriate
Reviewer 2 is @trinker (due date: by March 7th)
Please check off boxes as applicable, and elaborate in comments below. Your review is not limited to these topics, as described in the reviewer guide
The package includes all the following forms of documentation:
URL
, BugReports
and Maintainer
(which may be autogenerated via Authors@R
).For packages co-submitting to JOSS
- [ ] The package has an obvious research application according to JOSS's definition
The package contains a
paper.md
matching JOSS's requirements with:
- [ ] A short summary describing the high-level functionality of the software
- [ ] Authors: A list of authors with their affiliations
- [ ] A statement of need clearly stating problems the software is designed to solve and its target audience.
- [ ] References: with DOIs for all those that have one (e.g. papers, datasets, software).
Estimated hours spent reviewing: 3
This package allows an R user to access Qualtrics survey data quickly and easily within R by using the Qualtrics API feature. As an avid Qualtrics user, I was pleasantly surprised with how quickly and seamlessly I was able to access survey data for analysis, because this is normally a tedious and time-consuming task. I tested the package using surveys of various:
Overall I encountered no difficulty in installing and using the package, even when testing complex surveys with high response rates.
As indicated above, a statement of need has not yet been provided by the package authors. From my perspective, this package addresses several major needs often encountered by analysts who use R to work with Qualtrics data. Speaking broadly, Qualtrics itself can be quite finnicky in how it handles survey response data outside of Qualtrics. The qualtRics package helps mitigate this through the following methods:
circumvents character encoding errors that occur when downloading Qualtrics data and then bringing it into R.
utilizes question labels as header names, thereby preventing unwieldy and poorly formatted column headers.
seamlessly allows the user to analyze data from multiple related surveys. I often have to write two (or more) surveys that are linked together through embedded data and query strings, and this package makes accessing and joining the data significantly more straightforward.
the automatic conversion of variables makes my heart happy, and I'm hopeful that this feature will eventually extend past single answer multiple choice questions.
Using devtools::test()
had three failures, all with the following error message:
test-getSurveys.R:4: error: (unknown) could not find function "with_mock_api"
The explanation on registering Qualtrics credentials and creating a configuration file are aces. Providing links on where to find the Qualtrics API token that lead to Qualtrics help documentation reinforces what Qualtrics should be providing support on vs. what the package authors can provide support on.
Help documentation and the vignette both provide good baseline information on the package and provide examples for the use of each function. I do recommend adding in line spacing to help improve the readability of the help documentation examples.
Please check off boxes as applicable, and elaborate in comments below. Your review is not limited to these topics, as described in the reviewer guide
The package includes all the following forms of documentation:
URL
, BugReports
and Maintainer
(which may be autogenerated via Authors@R
).The package contains a paper.md
matching JOSS's requirements with:
Estimated hours spent reviewing: 4.5 hrs
Thanks for providing this interface to the R community. I was able to get up and running pretty quickly which is kudos considering I had never used Qualtrics and that the package is an API interface which can be tricky to have a smooth user interface. I have broken down additional comments into sub headings.
The package was pretty intuitive and easy to use. There were a few cognitive sticking points as I worked through the Vignette, README, and Examples that are in the form of suggestions below:
Messages (messages, warnings, and errors) were detailed and had suggestions for what to do.
qualtRicsConfigFile
's message. It currently is 419 characters wide, making it difficult to read. Same is true of the error message in getSurveys
for bad url. This is true across many messages, warnings, and errors. Between the README and Vignette I was up and running quickly. The links to the Qualtrics page for the base_url
and institution specific url were very helpful. I did find the following points that were a little challenging in getting set up:
root_url
vs. Qualtrics' term base_url
could be confusing. It's nice to be consistent when possible. In this case base_url
seems to match what Qualtrics calls hostname
(The /API
doesn't seem to be required by qualtRics). root_url
parameter for registerOptions
or the '.qualtRics.yml' is not consistent with the Qualtrics description which ends the url with /API
, using this format described by Qualtrics in the package causes an error. Perhaps (a) automatically stripping /API
off in functions like getSurveys
or (b) better description of this in the documentation would assist the user. Examples ran as expected with correct parameters and tweaks (examples with a package like this are difficult to do out of the box because every user's experience will be different). Nice job with making as generic as possible. I did note the following difficulties and/or suggestions:
readSurvey
example has readSurvey("<YOUR-PATH-TO-CSV-FILE>")
. A minimal .csv file could easily be included in the inst
file and accessed via system.file
. This would make the examples more reproducible. metadata
examples uses a slightly different format (and more difficult to follow) for giving surveyId
(it uses id
which is a variable that doesn't exist) while other function examples use surveyId = surveys$id[6]
(or simular) which connects to the prior step in the example and easy to follow.This appeared to pass CRAN checks. It does violate the following requirements in the Writing R Ext.:
The package generally complies with ROpenSci's Packaging Guidelines (https://github.com/ropensci/onboarding/blob/master/packaging_guide.md). Code contains many comments for ease of understanding. There is little if any code duplication. I did note:
getSurvey
?getSurvey
we see api_token="<YOUR-API-TOKEN>", root_url="<YOUR-ROOT-URL>")
and verbose=TRUE
Testing things that are secret (like API keys) is difficult. This package runs as many tests as possible. I did note:
devtools::test
had three failures: could not find function "with_mock_api"
If I load library(httptest)
it works. This may just be ignorance on my part on usage. Travis-CI seems to know how to handle this so it's likely me.Thanks for the reviews! I expect to start incorporating your feedback at the end of the week, as I'm currently wrapping up a project for a client.
Thanks @JasperHG90! Looking forward to your revisions.
Started working on revisions. Expect to finish by Friday 23 March at latest.
Hello all,
Thank you for your reviews. They were very useful. I've addressed most of them in the code (see the development branch), and have placed several comments/questions at the bottom of this post.
Updates
root_url
, and changed to base_url
across code, examples, vignette and README. Internally, the package still uses root_url, but this will be changed to base_url later. Added examples of what the base url passed to qualtRics package should look like to README and vignette.metadata()
function and added examples for readSurvey()
function.getSurvey()
: Deprecated arguments that use snake case in favour of arguments that use camel case.Questions/comments
Function titles are lower cased in .Rd files (and roxygen). CRAN Ext. states "Title information for the Rd file. This should be capitalized and not end in a period..."
Could you provide a link/example?
@JasperHG90 Sorry for the lack of clarity on the lower cased titles:
CRAN policies:: I don't understand this comment:
What I mean is that cran states:
Function titles are lower cased in .Rd files (and roxygen). CRAN Ext. states "Title information for the Rd file. This should be capitalized and not end in a period..."
If you go to one of your files: https://github.com/JasperHG90/qualtRics/blob/master/R/getSurvey.R
We see:
' Export a survey and download into R
at line 17. This should not be lower cased but in title case:
' Export a Survey and Download into R
This will then transfer to the .Rd files when you roxygenize.
My thinking on the failed test is that it only fails for devtools::test
and since it works on Travis-CI and when I run a CRAN check I'd say this can be ignored. I wouldn't move httptest to Imports unless you're actually importing it. I'd appreciate others to weigh in here.
Thanks for your comments @trinker . I edited the titles.
@kierisi Hi Jesse! 👋 Could you please take a look at @JasperHG90's recent changes and see if you are able to check off a statement of need issue? And please let us know if the fixes and updates are ok by you for a sign off. 🙏
clicked and on-board for a sign off :rocket:
@trinker Do you have other concerns that remain unaddressed or are you happy to sign off as well?
@karthik happy to sign off as well
Approved! 🎉 Thank you for submitting and @kierisi and @trinker for thorough and timely reviews! 🙏
@JasperHG90
To-dos:
[ ] Transfer the repo to the rOpenSci organization under "Settings" in your repo. I have invited you to a team that should allow you to do so. You'll be made admin once you do.
[ ] Add the rOpenSci footer to the bottom of your README
[![ropensci_footer](https://ropensci.org/public_images/ropensci_footer.png)](https://ropensci.org)
Welcome aboard! We'd also love a blog post about your package, either a short-form intro to it (https://ropensci.org/tech-notes/) or long-form post with more narrative about its development. ((https://ropensci.org/blog/). If you are, @stefaniebutland will be in touch about content and timing.
To-dos:
[x] Transfer the repo to the rOpenSci organization under "Settings" in your repo. I have invited you to a team that should allow you to do so. You'll be made admin once you do.
[x] Add the rOpenSci footer to the bottom of your README
[x] Fix any links in badges for CI and coverage to point to the ropensci URL. (We'll turn on the services on our end as needed)
Happy to write a technical note!
@JasperHG90 Great to hear that you'd like to publish a technote on this. We can publish at any time (in contrast to blog posts that are scheduled). Please submit a draft by pull request at your convenience and we can review before publishing.
Here are examples: https://ropensci.org/technotes/
Instructions on pull request and preview: https://github.com/ropensci/roweb2#contributing-a-blog-post. The only difference in YAML for technote is categories:
I will add the topicid before publishing.
Don't hesitate to ask any questions here.
Hi Stefanie. Thanks for the information!
@JasperHG90 One quick question. Would you be interested in adding @kierisi and @trinker as reviewers in your description file? Read more about why here
Also would be good to see if they're both comfortable with the idea.
e.g. https://github.com/ropensci/rdflib/blob/master/DESCRIPTION#L8-L10
fine by me!
What a neat feature! If @Kierisi and @trinker are OK, i’d Like to add them as reviewers. Please leave information in a comment so I can add!
On Wed, 11 Apr 2018 at 19:31, Karthik Ram notifications@github.com wrote:
@JasperHG90 https://github.com/JasperHG90 One quick question. Would you be interested in adding @kierisi https://github.com/kierisi and @trinker https://github.com/trinker are reviewers in your description file? Read more about why here https://ropensci.org/blog/2018/03/16/thanking-reviewers-in-metadata/
Also would be good to see if they're both comfortable with the idea.
e.g. https://github.com/ropensci/rdflib/blob/master/DESCRIPTION#L9-L11
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ropensci/onboarding/issues/192#issuecomment-380533412, or mute the thread https://github.com/notifications/unsubscribe-auth/AHtJuw8OnBE_Trr4GLu2q0Au5dlyneVzks5tnj3TgaJpZM4R4hH5 .
@kierisi @trinker If you don't already have an ORCID, sign up for one here: https://orcid.org/
here you go: 0000-0003-1915-9612
0000-0002-4299-7692
@karthik had one more question: could you explain to me how the JOSS process works? Is the accompanying paper automatically submitted or do I need to do something? Thanks!
@JasperHG90 Sure. Just follow the submission instructions for JOSS (http://joss.theoj.org/papers/new) and submit a paper there. You'll need to write the paper of course, and include some high level references in a .bib
file, then commit both to this same repo.
Then tag me as editor. I'll do a quick review of the paper itself and make sure it reads well. Once that's cleared, you will have to archive a copy of your repo in Zenodo (zenodo.org) and share the DOI. After that the EIC and I can finish the final steps for acceptance.
@JasperHG90 I'm going to close this issue here but feel free to ping me over email (see my profile) for help with the JOSS part. 🙏
Paper is now published on JOSS! 🎉 http://joss.theoj.org/papers/10.21105/joss.00690
@JasperHG90 Are you still interested/ willing to write a tech note about qualtRics
? I think it would really be of interest and would be great to bring some attention to the package
Hi Stefanie,
Thanks for your message. I'd be happy to do this but I am currently involved in a couple of projects that are taking up all of my time. Things should ease up around half August. I'll put this on the to-do list for around that time!
Best,
Jasper.
I'll ping you here in late August then @JasperHG90 so we can come up with a tentative date for a draft. Cheers
@JasperHG90 Hoping your projects load allows you some breathing room soon. I have blog post publication dates available Sept 18, Oct 2, 9, 16. If you're still interested, let me know your preferred date and mark your calendar to submit a draft a week in advance. Goal is to get more eyes on your work, but I know this is extra work beyond the review process. No worries if you're reluctant to spend the extra time.
Hi @JasperHG90. One last ping to ask if you want to write a qualtRics
post.
Summary
This package focuses on the retrieval of survey data using the Qualtrics API and aims to reduce the pre-processing steps needed to prepare this data for analysis. It also supports the import of manual data exports and other metadata (survey flow, questions, number of responses etc.).
https://github.com/JasperHG90/qualtRics
I would categorise it under 'data retrieval', since the package allows users to retrieve data exports from Qualtrics and import this data directly into R.
Target audience are analysts who use Qualtrics to design and disseminate surveys and who want straightforward access to their data using the API.
Not entirely. There are several packages that provide functionality to interact with the Qualtrics API, e.g.
However, I'm not sure if these packages are still under active development.
qualtRics
focuses entirely on importing survey data and attempts to make this as simple as possible. Furthermore, it is the only package that uses survey metadata to automatically convert survey questions.No pre-submission, but I was contacted by one of your colleagues who invited me to submit the package. See here
Requirements
Confirm each of the following by checking the box. This package:
Publication options
Already on CRAN. See CRAN page
paper.md
matching JOSS's requirements with a high-level description in the package root or ininst/
.Detail
R CMD check
(ordevtools::check()
) succeed? Paste and describe any errors or warnings:No warnings / errors
Yes with following notes:
Function names: these do not conform to the rOpenSci standards. I chose to keep them as is since this package has already been on CRAN for some time and changing the names of the functions would result in useless existing code for users.
News file: Until recently I used a Changelog.md file for news about changes. I'm now using rOpenSci headers for new releases, but have not done so for previous releases.
If this is a resubmission following rejection, please explain the change in circumstances:
If possible, please provide recommendations of reviewers - those with experience with similar packages and/or likely users of your package - and their GitHub user names: