search

cloudyr / cloudyr.github.io

the cloudyr project website
https://cloudyr.github.io
29 stars 5 forks source link

A cloudyr Qualtrics survey package #13

Closed jamesdunham closed 6 years ago

jamesdunham commented 7 years ago

I've written a toolkit for working with the Qualtrics survey platform and its data in R. High-level functionality focuses on testing and review of surveys before fielding, and analysis of responses afterward, at least for now. There are lower-level functions for working directly with the rest of the Qualtrics API.

Would this be a good fit for cloudyr? You can find docs at https://jdunham.io/qsurvey/.

leeper commented 7 years ago

Yes, this would definitely fit. It would be even better if we could collaborate to create 1 Qualtrics-related package rather than proliferating a bunch of rival packages. Maybe @JasperHG90 or @saberry would be interested?

JasperHG90 commented 7 years ago

Happy to talk about collaboration and merging our projects into 1 package!

saberry commented 7 years ago

I would be honored to collaborate with you all -- one Qualtrics-related package would be perfect.

On Tue, Jan 17, 2017 at 5:33 AM, Jasper Ginn notifications@github.com wrote:

Happy to talk about collaboration and merging our projects into 1 package!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/cloudyr/cloudyr.github.io/issues/13#issuecomment-273097673, or mute the thread https://github.com/notifications/unsubscribe-auth/AGX7ElSrYGtQIOLJJpJhfa_VJ4D-Q3pDks5rTJkEgaJpZM4LlCt8 .

-- Seth A. Berry, Ph.D. Research Analyst Mendoza College of Business University of Notre Dame 321 Mendoza Notre Dame, IN 46556

leeper commented 7 years ago

I don't have time at the moment to look through all the packages, but maybe one of you could take the lead on trying to identify common functionality as well as unique features and see what a reasonable route to integration might be?

I can fire up a repo here at cloudyr and give you all write access if you want to work in a common place.

jamesdunham commented 7 years ago

Terrific, I'd be very happy to work together on this project. Thanks @leeper. Here are some notes on common functionality and unique features.

These are initial thoughts to begin discussion. I'm looking forward to working together.

saberry commented 7 years ago

A most excellent rundown, James, and thanks for the links -- great seeing everything

I will do some exploring/thinking and circle back around with any thoughts.

On Tue, Jan 17, 2017 at 10:48 AM, James Dunham notifications@github.com wrote:

Terrific, I'd be very happy to work together on this project. Thanks @leeper https://github.com/leeper. Here are some notes on common functionality and unique features.

-

Necessarily each package has a solution to authentication. qsurvey uses environment variables by default, but each function that makes a request (example https://github.com/jamesdunham/qsurvey/blob/master/R/surveys.R) also takes arguments for setting the API key and subdomain in the call. To reduce barriers to this approach for users unfamiliar with environment variables, there's a function for setting them from a file. I see file-based approaches in both qualtRics and and qualtricsR. I'm sure qsurvey's implementation of this can be improved.

More generally, we all send requests to the API. Each package uses httr. @JasperHG90 https://github.com/JasperHG90, qualtRics handles https://github.com/JasperHG90/qualtRics/blob/master/R/qualtricsResponseCodes.R response codes in a particularly user-friendly way.

qualtricsR uniquely offers survey creation in R/RStudio and import into Qualtrics. Programmatic survey creation strikes me as fantastically useful.

We've each https://github.com/jamesdunham/qsurvey/blob/master/R/responses.R worked https://github.com/JasperHG90/qualtRics/blob/master/R/getSurvey.R on https://github.com/saberry/qualtricsR/blob/master/R/importQualtricsData.R downloading survey responses. @JasperHG90 https://github.com/JasperHG90, qualtRics uniquely allows downloading responses to a file in any of the supported formats (e.g. SPSS). qsurvey just reads them into R, though there's some code overlap because to do this it downloads them to a tempfile. I see value in allowing users to do either easily, so we might refactor into 1) common code for handling the POST and response; 2) a function for reading into R; and 3) a function for saving to disk. What do you both think?

These are initial thoughts to begin discussion. I'm looking forward to working together.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/cloudyr/cloudyr.github.io/issues/13#issuecomment-273207248, or mute the thread https://github.com/notifications/unsubscribe-auth/AGX7EpHwScH5soa65aTOofdt6qYgWvttks5rTOLBgaJpZM4LlCt8 .

-- Seth A. Berry, Ph.D. Research Analyst Mendoza College of Business University of Notre Dame 321 Mendoza Notre Dame, IN 46556

JasperHG90 commented 7 years ago

@jamesdunham

Necessarily each package has a solution to authentication. qsurvey uses environment variables by default, but each function that makes a request (example) also takes arguments for setting the API key and subdomain in the call. To reduce barriers to this approach for users unfamiliar with environment variables, there's a function for setting them from a file. I see file-based approaches in both qualtRics and and qualtricsR. I'm sure qsurvey's implementation of this can be improved.

I like your implementation. I have two thoughts on this: 1) environment variables look scary so I guess it'd be easy/user friendly to write a wrapper function like registerApiFunction.r in qualtRics, and 2) ideally, users would only have to register their API key once. I've seen applications in python that save the API key credentials as a pickle file in a hidden folder in the user's home directory. However, I think CRAN's policies prohibits us from changing files on disk except for the temporary folder.

More generally, we all send requests to the API. Each package uses httr. @JasperHG90, qualtRics handles response codes in a particularly user-friendly way.

Yes, I suppose we'd take the cleanest approach. I believe qsurvey fits this definition.

qualtricsR uniquely offers survey creation in R/RStudio and import into Qualtrics. Programmatic survey creation strikes me as fantastically useful.

Agreed.

We've each worked on downloading survey responses. @JasperHG90, qualtRics uniquely allows downloading responses to a file in any of the supported formats (e.g. SPSS). qsurvey just reads them into R, though there's some code overlap because to do this it downloads them to a tempfile. I see value in allowing users to do either easily, so we might refactor into 1) common code for handling the POST and response; 2) a function for reading into R; and 3) a function for saving to disk. What do you both think?

I'm currently re-writing this part of the code. When a user downloads surveys from Qualtrics (e.g. in CSV format), the survey data contains no useful factors / data types and doesn't know how to deal with labels, scoring and other advanced features. I'm using the survey information to add this information, infer the right factor levels / data types, and to return an altogether more 'clean' survey result.

jamesdunham commented 7 years ago

Thanks @JasperHG90 for the detailed comments.

authentication

1) environment variables look scary so I guess it'd be easy/user friendly to write a wrapper function like registerApiFunction.r in qualtRics,

I agree, environment variables would be a barrier for some users. So, a function that eases the use of API credentials seems important. This function would ideally address the next point you raise about users needing to provide their key anew in every R session.

It could be helpful to describe the use of such a function with installation/setup instructions. One solution: we could guide users to:

  1. Retrieve/generate their API key from Qualtrics (unavoidable; we can provide instructions)
  2. Save it in a file whose default location we'd suggest, something like ~/.qualtrics_key (permissions recommended)
  3. Provide a function that reads a key file and sets the environment variable, which could be called at the start of scripts and would by default look in the suggested location

So, after the initial setup costs, users would just need library(qualtrics); set_key() or whatever. (If they don't already use environment variables.) Thoughts?

response exports

I'm currently re-writing this part of the code. When a user downloads surveys from Qualtrics (e.g. in CSV format), the survey data contains no useful factors / data types and doesn't know how to deal with labels, scoring and other advanced features. I'm using the survey information to add this information, infer the right factor levels / data types, and to return an altogether more 'clean' survey result.

That's exciting; I think this would be incredibly useful. And, it's functionality that would make the package very helpful to a set of folks who would otherwise make do with the GUI.

next steps

I don't want to move ahead before giving @saberry a chance to reply but I can see a few good ways to proceed. Here's a possible division of responsibilities:

Practically, we could start from the cloudyr package template, and then add these pieces:

I'd be happy to set up the template.

Two last notes are that we should each reread the cloudyr style guide, and it would be terrific to take advantage of good test coverage from the outset. @leeper, would you start a repo for us to work in?

Thanks.

saberry commented 7 years ago

That all sounds fantastic and I like the proposed setup solution.

It is great to hear that you are working on improving the response export, Jasper!

I will start going through the style guide and messing with the qualtricsR stuff.

Thanks again for everyone's work on this,

Seth

On Thu, Jan 19, 2017 at 10:42 AM, James Dunham notifications@github.com wrote:

Thanks @JasperHG90 https://github.com/JasperHG90 for the detailed comments. authentication

  1. environment variables look scary so I guess it'd be easy/user friendly to write a wrapper function like registerApiFunction.r in qualtRics,

I agree, environment variables would be a barrier for some users. So, a function that eases the use of API credentials seems important. This function would ideally address the next point you raise about users needing to provide their key anew in every R session.

It could be helpful to describe the use of such a function with installation/setup instructions. One solution: we could guide users to:

  1. Retrieve/generate their API key from Qualtrics (unavoidable; we can provide instructions)
  2. Save it in a file whose default location we'd suggest, something like ~/.qualtrics_key (permissions recommended)
  3. Provide a function that reads a key file and sets the environment variable, which could be called at the start of scripts and would by default look in the suggested location

So, after the initial setup costs, users would just need library(qualtrics); set_key() or whatever. (If they don't already use environment variables.) Thoughts? response exports

I'm currently re-writing this part of the code. When a user downloads surveys from Qualtrics (e.g. in CSV format), the survey data contains no useful factors / data types and doesn't know how to deal with labels, scoring and other advanced features. I'm using the survey information to add this information, infer the right factor levels / data types, and to return an altogether more 'clean' survey result.

That's exciting; I think this would be incredibly useful. And, it's functionality that would make the package very helpful to a set of folks who would otherwise make do with the GUI. next steps

I don't want to move ahead before giving @saberry https://github.com/saberry a chance to reply but I can see a few good ways to proceed. Here's a possible division of responsibilities:

  • I'd be happy to take the lead on implementing and documenting a friendly solution to authentication.
  • @JasperHG90 https://github.com/JasperHG90, you could take charge of response exports using whatever combination of existing code you think best
  • @saberry https://github.com/saberry, you could start a branch that would simply integrate what's already in qualtricsR, since it's unique.

Practically, we could start from the cloudyr package template https://github.com/cloudyr/pkgtemplate, and then add these pieces:

I'd be happy to set up the template.

Two last notes are that we should each reread the cloudyr style guide https://cloudyr.github.io/styleguide/, and it would be terrific to take advantage of good test coverage from the outset. @leeper https://github.com/leeper, would you start a repo for us to work in?

Thanks.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/cloudyr/cloudyr.github.io/issues/13#issuecomment-273810099, or mute the thread https://github.com/notifications/unsubscribe-auth/AGX7Eq3g947KdDg2iYa7T30gCk0aP181ks5rT4RXgaJpZM4LlCt8 .

-- Seth A. Berry, Ph.D. Research Analyst Mendoza College of Business University of Notre Dame 321 Mendoza Notre Dame, IN 46556

leeper commented 7 years ago

Great. Let me know if you want a fresh repo, or if you want to migrate one of the existing repos and jump off from there.

In general, I think environment variables are the right way to store credentials because that enables "power users" to handle them outside of R if they want and it allows for secure testing on Travis without disclosing keys accidentally. Convenience functions to help set/retrieve them are a good way to make that functionality available to users who might not be comfortable setting them up.

A reasonable configuration might be:

jamesdunham commented 7 years ago

Thanks @leeper. That seems reasonable to me.

Let's start from the cloudyr template.

leeper commented 7 years ago

Okay, I've copied the template into a new repo (https://github.com/cloudyr/qualtrics), added you all to a development team, and given you admin access to that repo.

JasperHG90 commented 7 years ago

Thanks @leeper .

@jamesdunham: like and agree with your outline!

leeper commented 6 years ago

Looks like a stable project has moved forward here: https://github.com/ropensci/qualtRics so I'm closing this issue.