include_questions not working

[ccobserver]

When I run the following code, I'm able to pull in all of the data for a particular survey:

mysurvey <- fetch_survey(surveyID = '')

But when I try to narrow it down to a few specific questions using the below code, it doesn't work

mysurvey <- fetch_survey(surveyID = '', include_questions = c('Q1', 'Q2'))

The survey I'm interested in viewing has over 1 million responses and hundreds of questions, so it isn't practical to download the entire survey and filter to the columns needed.

Am I doing something wrong with this fetch_survey option?

[kevintroy]

The endpoint used by fetch_survey doesn’t use the user-facing question numbers (like “Q1”). Rather, it uses a backend question ID, which will look something like “QID1.” (You can re-number questions to change the user-facing question numbers, but you can’t change the backend question IDs — they’re the primary keys of the table of questions.)

You can get a mapping of front end question number to backend question ID by using the survey_questions() function.

[ccobserver]

Thanks, Kevin. I tried using both qname and qid from survey_questions() Neither works. I get the same error message:

Error in qualtrics_response_codes(res) : Qualtrics API raised a bad request (400) error - Please report this on https://github.com/ropensci/qualtRics/issues

I'm not sure where to go from here.

On Mon, Mar 23, 2020 at 6:15 PM Kevin Troy notifications@github.com wrote:

The endpoint used by fetch_survey doesn’t use the user-facing question numbers (like “Q1”). Rather, it uses a backend question ID. (You can re-number questions to change the user-facing question numbers, but you can’t change the backend question IDs — they’re the primary keys of the table of questions.)

You can get a mapping of front end question number to backend question ID by using the survey_questions() function.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/ropensci/qualtRics/issues/147#issuecomment-602908136, or unsubscribe https://github.com/notifications/unsubscribe-auth/AC3NLW7PLHQPXHUZDXWIYADRI7UPJANCNFSM4LSGLHIQ .

[kevintroy]

And you're quite sure that the surveyID is correct? This sounds silly, but: when you added the include_questions parameter to your code, did you accidentally change the surveyID parameter (e.g. adding an extra character on the end)? The 400 error you're getting will be raised by Qualtrics if you're asking for a surveyID that doesn't exist. (It will be raised by Qualtrics in other cases too, but that's one of them.) If you get the surveyID right, and ask for a questionID that doesn't exist in the include_questions parameter, you should get a 404 "resource not found" error.

Beyond that, I'm out of ideas. What version of the package are you running?

[juliasilge]

I was able to reproduce this problem; the name in the JSON payload that was being sent to the Qualtrics API had not been updated to the new API.

Can you install from GitHub and try again? I believe this should solve the problem.

devtools::install_github("ropensci/qualtRics")

Thank you so much for the report! 🙌

[ccobserver]

I get the same error when I include a list of QIDs - include_questions = c('QID2', 'QID3')

But it works with just one QID selected - include_questions = 'QID2'

Could it now be an issue with json formatting for a list/array?

[juliasilge]

Hmmmm, I don't think so; it works with multiple questions if I make sure they are the actual QIDs:

library(qualtRics)
fetch_survey("SV_XXXXXXX", 
             include_questions = c("QID18", "QID19"))
#>   |                                                                              |                                                                      |   0%  |                                                                              |======================================================================| 100%
#> Parsed with column specification:
#> cols(
#>   .default = col_character(),
#>   StartDate = col_datetime(format = ""),
#>   EndDate = col_datetime(format = ""),
#>   Progress = col_double(),
#>   `Duration (in seconds)` = col_double(),
#>   Finished = col_logical(),
#>   RecordedDate = col_datetime(format = ""),
#>   RecipientLastName = col_logical(),
#>   RecipientFirstName = col_logical(),
#>   RecipientEmail = col_logical(),
#>   ExternalReference = col_logical(),
#>   LocationLatitude = col_double(),
#>   LocationLongitude = col_double(),
#>   SolutionRevision = col_double()
#> )
#> See spec(...) for full column specifications.
#> # A tibble: 26 x 20
#>    StartDate           EndDate             Status IPAddress Progress
#>    <dttm>              <dttm>              <chr>  <chr>        <dbl>
#>  1 2020-02-20 01:16:42 2020-02-20 01:17:19 Surve… <NA>           100
#>  2 2020-02-20 01:30:55 2020-02-20 01:34:37 IP Ad… 98.14.36…      100
#>  3 2020-02-20 01:49:48 2020-02-20 01:50:23 IP Ad… 75.82.50…      100
#>  4 2020-02-20 02:46:41 2020-02-20 02:47:01 IP Ad… 75.172.1…      100
#>  5 2020-02-20 02:56:28 2020-02-20 02:59:15 IP Ad… 194.59.2…      100
#>  6 2020-02-20 12:22:10 2020-02-20 12:22:58 IP Ad… 66.168.1…      100
#>  7 2020-02-20 12:31:28 2020-02-20 12:32:06 IP Ad… 35.138.9…      100
#>  8 2020-02-20 12:52:59 2020-02-20 12:53:26 IP Ad… 24.254.1…      100
#>  9 2020-02-20 17:09:48 2020-02-20 17:10:12 IP Ad… 99.88.19…      100
#> 10 2020-02-21 02:52:01 2020-02-21 02:52:02 Surve… <NA>           100
#> # … with 16 more rows, and 15 more variables: `Duration (in seconds)` <dbl>,
#> #   Finished <lgl>, RecordedDate <dttm>, ResponseId <chr>,
#> #   RecipientLastName <lgl>, RecipientFirstName <lgl>, RecipientEmail <lgl>,
#> #   ExternalReference <lgl>, LocationLatitude <dbl>, LocationLongitude <dbl>,
#> #   DistributionChannel <chr>, UserLanguage <chr>, Q18 <ord>, Q19 <chr>,
#> #   SolutionRevision <dbl>

^{Created on 2020-03-25 by the reprex package (v0.3.0)}

This is a survey that has many more questions.

Can you double check what is going on with your QIDs?

[ccobserver]

It must have had something to do with the second question I was pulling in. I tried with a separate survey and all is well. I really appreciate your help! I love this package!

[juliasilge]

Sounds good, and thanks so much! I'm going to call this closed in 6f5c0c5e39657464eb2402493a8d730643b0c805.