Add survey-specific filters to fetch_survey()

[markjrieke]

Is it possible to add survey-specific filters to fetch_survey()? I can filter after fetching but this would help avoid loading unnecessary info into the environment (helpful when dealing with a huge number of responses).

For example, I work with a hospital system & often need to get a specific campus' responses. Currently, I can do this with:

fetch_survey(surveyID) %>% filter(CAMPUS == campusname)

where CAMPUS is one of the columns from fetch_survey(). It'd be great to apply an arbitrary number of survey-specific filters during the fetch_survey() call.

[juliasilge]

The fetch_survey() function uses the export-responses endpoint of the Qualtrics API. We can't pass in arbitrary dplyr-style filtering or other operations, but the Qualtrics API does have concept of a filter than you define in the Qualtrics UI on a given survey. I don't think we can currently support that but we should consider it, and/or try to revamp these functions to make them more flexible to accepting different arguments like these.

[kevintroy]

@juliasilge I have some free time this coming week and could take a crack at this. AFAICT, would need to create a function that pulls data down from the "get list of available filters" endpoint, so the user knows which filterID to pass to the API. And of course would need to modify fetch_survey() to accept the filterID parameter.

[jmobrien]

@kevintroy a minimal change that would address @markjrieke's needs would be to just add filterId to `fetch_survey() as you mentioned, which then gets added to the subsequent query (after the usual param checks). The docs for get list of available filters are clear that all filters have to be set up on the web interface anyway, so getting the filter ID's from there would be quite possible.

IMO this would be a great place to add something to the package with a high ROI, with something for getting a survey's filters a great second act (once we get the URL generation thing fully settled.)

(FYI this will need some mocked tests, too. Here's some great resources @juliasilge shared with me about that, while #200 and #201 get into some of the details about how we're making mocked tests work for this package).

[aaronball85]

I'd like to add my interest in seeing a filter_id argument added in the fetch_survey() function as a minimal change approach as suggested by @jmobrien. This could be a high value add, low cost feature addition.

What I believe is involved: 1) adjust fetch_survey() function to include a filter_id argument (e.g. filter_id = NULL) 2) perhaps add a checkarg_isstring() function to validate that the supplied filter_id is a string 3) modify the create_raw_payload() function argument set to include filterID = filter_id to be included in raw_payload which is passed to the body argument of export_responses_request().

Any possibility this functionality could be added in the near-term future?

[jmobrien]

I'd add that we'd want to immediately have clear documentation that the web UI is required for using this feature, since there's no API-only approach for defining a filter.

If this feature is added, we should probably follow up any add with a new list_survey_filters() or similarly named function that could list available filters for any given survey.

@aaronball85 seems like you have a decent handle on what would be needed for adding the feature; do you want to take a stab at it?