Closed flavioazevedo closed 5 years ago
I am so sorry for the frustration you have experienced @flavioazevedo! Let's see if we can get to the bottom of it.
If you are open to this, can you run the following code and then email me (my address is in the DESCRIPTION file) the CSV file at the path you find? It will be a fairly raw, unprocessed CSV file, but that is the one that is causing problems with readr::read_csv()
, apparently!
root_url <- qualtRics:::append_root_url(Sys.getenv("QUALTRICS_BASE_URL"), "responseexports")
raw_payload <- qualtRics:::create_raw_payload(
surveyID = <YOUR SURVEY ID HERE>,
label = TRUE,
last_response = NULL,
start_date = NULL,
end_date = NULL,
unanswer_recode = NULL,
limit = NULL,
local_time = FALSE,
include_questions = NULL
)
res <- qualtRics:::qualtrics_api_request("POST", url = root_url, body = raw_payload)
ID <- res$result$id
survey.fpath <- qualtRics:::download_qualtrics_export(paste0(root_url, ID), verbose = TRUE)
survey.fpath
What happens if you use convert = FALSE
in fetch_survey()
?
The other thing I would love to know is if the output you get after the parsing failures is actually garbled. What happens here is that R sends a request to the Qualtrics API and the API sends back a zipped CSV file. Then R unzips the CSV file and reads it using readr::read_csv()
. There are in fact sometimes parsing failures, because the CSVs are sometimes malformed or having something weird in them. The good thing about readr::read_csv()
is that is does the best it can and usually returns something pretty usable, and then there is code in the qualtRics package to clean up the results. You have experienced these parsing failures, but is the output in fact not sensible?
I'm also seeing this.. I can report that, at least in my case, the affected columns are all NA, when I can see on the Qualtrics website that in the original data only ~50% had missing values.
I ran your code, and I can see the data just fine. Unfortunately, I'm unable to send you any data, for ethical reasons.
Any help would be greatly appreciated. I haven't used this code in a long time and I don't recall ever having this issue in previous versions (of my code, or the package).. Thanks, Michael
Also, when manually downloading the data as CSV and using read_survey()
, all seems OK.
I've also got this issue today. It appears to uniquely affect items where one of the options is "Not listed (please specify)" with an associated text field as the next variable. I'll have to check my ethical approval, but I may be able to share the csv file (or at least a portion of it). Cheers, Mathew
I made some changes this evening that should make data import more robust. If you get a chance, reinstall this package from GitHub and see if this solves some of the parsing issues you have been seeing:
install.packages("remotes")
remotes::install_github("ropensci/qualtRics")
Thanks so much! 🙌
Unfortunately it's still breaking in the same way, Julia. If it will help, I can send you extracts of the data, I will just have to excise some variables to minimise identifiability risks. Just let me know what you need and in what formats.
😭 😭 😭
Well, let's try some other things. If you are able, can you run the following code and then email me (my address is in the DESCRIPTION file) the CSV file at the path you find? It will be a fairly raw, unprocessed CSV file, but that is the one that is causing problems with data import.
root_url <- qualtRics:::append_root_url(Sys.getenv("QUALTRICS_BASE_URL"), "responseexports")
raw_payload <- qualtRics:::create_raw_payload(
surveyID = <YOUR SURVEY ID HERE>,
label = TRUE,
last_response = NULL,
start_date = NULL,
end_date = NULL,
unanswer_recode = NULL,
limit = NULL,
local_time = FALSE,
include_questions = NULL
)
res <- qualtRics:::qualtrics_api_request("POST", url = root_url, body = raw_payload)
ID <- res$result$id
survey.fpath <- qualtRics:::download_qualtrics_export(paste0(root_url, ID), verbose = TRUE)
survey.fpath
Looks like this issue is now resolved. To do the conversion to factors, we need the labels from Qualtrics. This means that label = FALSE
needs to be used with convert = FALSE
.
fetch_survey(my_id, label = FALSE, convert = FALSE)
I've added some new condition checking/error messaging.
If you continue to have problems, open up a new issue with the details!
I am having a parsing issue while using your package QualtRics. Basically, it shows anywhere from 500 parsing issues to 6000 (on other datas I tried to download from the Qualtrics website via your R package. I would love if this was just me being stupid, but before contacting you I have tried to solve for 2 days already. Below you find the error I am given. I am using R 3.6 and latest RStudio. I replicated results in base R, just in case it was something related to RStudio (had that before). I am installing the qualtRics directly from your GitHub repository.
Hopefully, you have a solution. Thank you so much for your work!