Closed cmbohn closed 7 years ago
Thanks for your comment.
When importing survey data into R, is there a delay between what is current in the Qualtrics system and what is being downloaded?
I'm not aware of such a delay. Essentially, you're just making a call to the Qualtrics API in which you request a csv file of the response exports.
Is it possible that the data is having problems importing?
Yes, that is possible. (see e.g. #13). In this case, I'm guessing there are special characters in your csv file. I googled that error and found the following:
As you can see here, the readSurvey()
function uses UTF-8 for the 'fileEncoding' argument.
Here's what I suggest you try:
# Get temp dir
td <- tempdir()
# List files and show
list.files(td)
# Copy the name of your survey
file_name <- paste(td, "/", "<YOUR-SURVEY>.csv")
# Read it into R
surv <- read.csv(file=file_name,
header = FALSE,
sep = ',',
stringsAsFactors=FALSE,
fileEncoding = "latin1",
skip = 3)
Best,
J.
The code provided isn’t connecting.
I have tried a number of variations for inputting the file name:
file_name <- paste(td, "/", "
All attempts have resulted in the system not being able to find the file or directory. The temp directory did not list a csv file. So I tried entering the files that exist in the temp file. Below is just one example of the same results I received.
Error in file(file, "rt", encoding = fileEncoding) : cannot open the connection In addition: Warning message: In file(file, "rt", encoding = fileEncoding) : cannot open file 'C:\Users\cb37543\AppData\Local\Temp\RtmpKiJALO / SV_5nIBD5iRalXmdNz.rds': No such file or directory
Chad
From: Jasper Ginn [mailto:notifications@github.com] Sent: Wednesday, June 28, 2017 1:01 AM To: JasperHG90/qualtRics qualtRics@noreply.github.com Cc: Bohn, Chad Chad.Bohn@jetblue.com; Author author@noreply.github.com Subject: Re: [JasperHG90/qualtRics] Data imported to R is not current (#36)
Thanks for your comment.
When importing survey data into R, is there a delay between what is current in the Qualtrics system and what is being downloaded?
I'm not aware of such a delay. Essentially, you're just making a call to the Qualtrics API in which you request a csv file of the response exports.
Is it possible that the data is having problems importing?
Yes, that is possible. (see e.g. #13https://github.com/JasperHG90/qualtRics/issues/13). In this case, I'm guessing there are special characters in your csv file. I googled that error and found the following:
As you can see herehttps://github.com/JasperHG90/qualtRics/blob/master/R/readSurvey.R#L46-L51, the readSurvey() function uses UTF-8 for the 'fileEncoding' argument.
Here's what I suggest you try:
td <- tempdir()
list.files(td)
file_name <- paste(td, "/", "
surv <- read.csv(file=file_name,
header = FALSE,
sep = ',',
stringsAsFactors=FALSE,
fileEncoding = "latin1",
skip = 3)
Best,
J.
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/JasperHG90/qualtRics/issues/36#issuecomment-311574567, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AcXCVYpfPkJIy5Z3bwJHmD_ONDNWivlaks5sIfpDgaJpZM4OG_o9.
Hi chad,
I didn't realize you're on a windows machine. Try:
file_name <- paste(td, "\", "
On 28 Jun 2017, 18:15 +0200, cmbohn notifications@github.com, wrote:
The code provided isn’t connecting. I have tried a number of variations for inputting the file name: file_name <- paste(td, "/", "
.csv") All attempts have resulted in the system not being able to find the file or directory. The temp directory did not list a csv file. So I tried entering the files that exist in the temp file. Below is just one example of the same results I received.
Error in file(file, "rt", encoding = fileEncoding) : cannot open the connection In addition: Warning message: In file(file, "rt", encoding = fileEncoding) : cannot open file 'C:\Users\cb37543\AppData\Local\Temp\RtmpKiJALO / SV_5nIBD5iRalXmdNz.rds': No such file or directory
Chad
From: Jasper Ginn [mailto:notifications@github.com] Sent: Wednesday, June 28, 2017 1:01 AM To: JasperHG90/qualtRics qualtRics@noreply.github.com Cc: Bohn, Chad Chad.Bohn@jetblue.com; Author author@noreply.github.com Subject: Re: [JasperHG90/qualtRics] Data imported to R is not current (#36)
Thanks for your comment.
When importing survey data into R, is there a delay between what is current in the Qualtrics system and what is being downloaded?
I'm not aware of such a delay. Essentially, you're just making a call to the Qualtrics API in which you request a csv file of the response exports.
Is it possible that the data is having problems importing?
Yes, that is possible. (see e.g. #13https://github.com/JasperHG90/qualtRics/issues/13). In this case, I'm guessing there are special characters in your csv file. I googled that error and found the following:
- Invalid Input causes read.csv to cut off datahttps://stackoverflow.com/questions/19610966/invalid-input-causes-read-csv-to-cut-off-data
- Invalid multibyte string in read.csvhttps://stackoverflow.com/questions/14363085/invalid-multibyte-string-in-read-csv?rq=1
- Reading in data with unusual charactershttps://stackoverflow.com/questions/19301265/reading-in-data-with-unusual-characters
As you can see herehttps://github.com/JasperHG90/qualtRics/blob/master/R/readSurvey.R#L46-L51, the readSurvey() function uses UTF-8 for the 'fileEncoding' argument.
Here's what I suggest you try:
- Re-run your code until you see the error
- Execute the following:
Get temp dir
td <- tempdir()
List files and show
list.files(td)
Copy the name of your survey
file_name <- paste(td, "/", "
.csv") Read it into R
surv <- read.csv(file=file_name,
header = FALSE,
sep = ',',
stringsAsFactors=FALSE,
fileEncoding = "latin1",
skip = 3)
- Let me know if that worked. If it does, I'll add 'fileEncoding' as an optional argument.
Best,
J.
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/JasperHG90/qualtRics/issues/36#issuecomment-311574567, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AcXCVYpfPkJIy5Z3bwJHmD_ONDNWivlaks5sIfpDgaJpZM4OG_o9. — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.
I thought of that too and tried it before this message. ☺
Because it is a windows machine, it actually needs 2 slashes.
file_name <- past(td, “\”, “
And unfortunately, I still got the same error and warning.
It isn’t a deal breaker or a problem for me to download the CSV from Qualtrics. That isn’t holding me up. But I would like to continue to help figure out what is going on.
From: Jasper Ginn [mailto:notifications@github.com] Sent: Wednesday, June 28, 2017 11:27 AM To: JasperHG90/qualtRics qualtRics@noreply.github.com Cc: Bohn, Chad Chad.Bohn@jetblue.com; Author author@noreply.github.com Subject: Re: [JasperHG90/qualtRics] Data imported to R is not current (#36)
Hi chad,
I didn't realize you're on a windows machine. Try:
file_name <- paste(td, "\", "
On 28 Jun 2017, 18:15 +0200, cmbohn notifications@github.com<mailto:notifications@github.com>, wrote:
The code provided isn’t connecting. I have tried a number of variations for inputting the file name: file_name <- paste(td, "/", "
.csv") All attempts have resulted in the system not being able to find the file or directory. The temp directory did not list a csv file. So I tried entering the files that exist in the temp file. Below is just one example of the same results I received.
Error in file(file, "rt", encoding = fileEncoding) : cannot open the connection In addition: Warning message: In file(file, "rt", encoding = fileEncoding) : cannot open file 'C:\Users\cb37543\AppData\Local\Temp\RtmpKiJALO / SV_5nIBD5iRalXmdNz.rds': No such file or directory
Chad
From: Jasper Ginn [mailto:notifications@github.com] Sent: Wednesday, June 28, 2017 1:01 AM To: JasperHG90/qualtRics qualtRics@noreply.github.com<mailto:qualtRics@noreply.github.com> Cc: Bohn, Chad Chad.Bohn@jetblue.com<mailto:Chad.Bohn@jetblue.com>; Author author@noreply.github.com<mailto:author@noreply.github.com> Subject: Re: [JasperHG90/qualtRics] Data imported to R is not current (#36)
Thanks for your comment.
When importing survey data into R, is there a delay between what is current in the Qualtrics system and what is being downloaded?
I'm not aware of such a delay. Essentially, you're just making a call to the Qualtrics API in which you request a csv file of the response exports.
Is it possible that the data is having problems importing?
Yes, that is possible. (see e.g. #13https://github.com/JasperHG90/qualtRics/issues/13). In this case, I'm guessing there are special characters in your csv file. I googled that error and found the following:
- Invalid Input causes read.csv to cut off datahttps://stackoverflow.com/questions/19610966/invalid-input-causes-read-csv-to-cut-off-data
- Invalid multibyte string in read.csvhttps://stackoverflow.com/questions/14363085/invalid-multibyte-string-in-read-csv?rq=1
- Reading in data with unusual charactershttps://stackoverflow.com/questions/19301265/reading-in-data-with-unusual-characters
As you can see herehttps://github.com/JasperHG90/qualtRics/blob/master/R/readSurvey.R#L46-L51, the readSurvey() function uses UTF-8 for the 'fileEncoding' argument.
Here's what I suggest you try:
- Re-run your code until you see the error
- Execute the following:
Get temp dir
td <- tempdir()
List files and show
list.files(td)
Copy the name of your survey
file_name <- paste(td, "/", "
.csv") Read it into R
surv <- read.csv(file=file_name,
header = FALSE,
sep = ',',
stringsAsFactors=FALSE,
fileEncoding = "latin1",
skip = 3)
- Let me know if that worked. If it does, I'll add 'fileEncoding' as an optional argument.
Best,
J.
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/JasperHG90/qualtRics/issues/36#issuecomment-311574567, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AcXCVYpfPkJIy5Z3bwJHmD_ONDNWivlaks5sIfpDgaJpZM4OG_o9. — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/JasperHG90/qualtRics/issues/36#issuecomment-311730469, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AcXCVWKlatVVExV5NGR7HoD_QdJ7JoLRks5sIoz_gaJpZM4OG_o9.
Thanks for helping out :-). It's useful for me to know such issues so I can solve them.
You are right: without specifying a directory in which to save the export the csv file is deleted from the temporary folder.
Can you follow the steps in the image below and tell me if this works for you?
Best,
J.
As an aside: is the data you download from Qualtrics manually and load into R complete?
Yes. The data appears to be complete (meaning that it has the number of rows in the CSV matches what is in Qualtrics). However, the way the CSV export is formatted it appears to be different from an API. As a result I have to do some recoding to the CSV dataset or accommodate the CSV format in my R code.
From: Jasper Ginn [mailto:notifications@github.com] Sent: Wednesday, June 28, 2017 1:35 PM To: JasperHG90/qualtRics qualtRics@noreply.github.com Cc: Bohn, Chad Chad.Bohn@jetblue.com; Author author@noreply.github.com Subject: Re: [JasperHG90/qualtRics] Data imported to R is not current (#36)
As an aside: is the data you download from Qualtrics manually and load into R complete?
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/JasperHG90/qualtRics/issues/36#issuecomment-311764468, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AcXCVdUpVAMyy_0Z6N0BjIsjsB-ofxhLks5sIqr-gaJpZM4OG_o9.
I ran the code you gave me below. It brought in more data but not the complete data. The complete data has around 1085 observations. This time around it only brought in 273 observations – 88 when I first contacted you. So there is that… I am still getting a warning too.
Warning messages: 1: In scan(file = file, what = what, sep = sep, quote = quote, dec = dec, : invalid input found on input connection 'C:\Users\cb37543\AppData\Local\Temp\RtmpKiJALO\Loyalty perceptions (Jamie).csv' 2: In scan(file = file, what = what, sep = sep, quote = quote, dec = dec, : EOF within quoted string
From: Jasper Ginn [mailto:notifications@github.com] Sent: Wednesday, June 28, 2017 1:29 PM To: JasperHG90/qualtRics qualtRics@noreply.github.com Cc: Bohn, Chad Chad.Bohn@jetblue.com; Author author@noreply.github.com Subject: Re: [JasperHG90/qualtRics] Data imported to R is not current (#36)
Thanks for helping out :-). It's useful for me to know such issues so I can solve them.
You are right: without specifying a directory in which to save the export the csv file is deleted from the temporary folder.
Can you follow the steps in the image below and tell me if this works for you?
[rconsole]https://user-images.githubusercontent.com/8079803/27656317-d21a9f18-5c48-11e7-81a7-484c89d9a526.png
Best,
J.
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/JasperHG90/qualtRics/issues/36#issuecomment-311763069, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AcXCVZs7pSgx1ZzwrOziiTI2YBklZRUrks5sIqmOgaJpZM4OG_o9.
Hm.
Last thing I suggest you try:
After downloading the csv in a temporary folder, copy-paste the temporary folder in your file manager and open de csv file in excel.
Two things I'd like to know about your survey:
Thanks for your help!
I don’t believe the open text responses contain any non-English characters. I haven’t found any.
The open text responses could contain a couple sentences in some cases. We limited the character count, but I do see some comments with about 2 sentences.
I notice when I open the temp file, there isn’t a copy of the csv in the temp folder. Only rds files. I tried taking a copy of the csv file that I downloaded directly from Qualtrics and put it in the temp folder. It didn’t recognize it and gave me the same warning message that I sent previously.
From: Jasper Ginn [mailto:notifications@github.com] Sent: Thursday, June 29, 2017 12:47 AM To: JasperHG90/qualtRics qualtRics@noreply.github.com Cc: Bohn, Chad Chad.Bohn@jetblue.com; Author author@noreply.github.com Subject: Re: [JasperHG90/qualtRics] Data imported to R is not current (#36)
Hm.
Last thing I suggest you try:
After downloading the csv in a temporary folder, copy-paste the temporary folder in your file manager and open de csv file in excel.
Two things I'd like to know about your survey:
Thanks for your help!
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/JasperHG90/qualtRics/issues/36#issuecomment-311877649, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AcXCVVJC4oiU4ADd9eMSHuhjLTFBithEks5sI0h7gaJpZM4OG_o9.
Ok. Thanks for trying out these suggestions. I'll close this issue for now because I can't reproduce it on my end, but I'll get back to this at a later point in time.
Issue should be solved. Please look at #38
When importing survey data into R, is there a delay between what is current in the Qualtrics system and what is being downloaded?
Is it possible that the data is having problems importing? I enter the following into R:
loyperc <- getSurvey("SV_5nIBD5iRalXmdNz", useLabels=FALSE, useLocalTime=TRUE, dateWarning=FALSE, convertStandardColumns=TRUE, force_request=TRUE)
I get the following warning:Warning message: In scan(file = file, what = what, sep = sep, quote = quote, dec = dec, : invalid input found on input connection 'C:/Users/cb37543/AppData/Local/Temp/RtmpeEgLjr/Loyalty perceptions (Jamie).csv'
I can only import 88 observations, when there is actually much more than this.