error when get _responses

[gitbobo]

get_session_key() [1] "ddrvaxfczqv9i5bhf4h34ukq5h86f4jt" responses <- get_responses(913762) Error in make.names(col.names, unique = TRUE) : invalid multibyte string 1

could you pls tell me what's the problem may be, thanks!

[r0berts]

I got the same error and then I looked at survey permissions - turned out that the user did not have survey permissions, once I added them I got data. Strange thing however was that I got all fields in one column. Troubleshooting that now.

[Jan-E]

@r0berts

Strange thing however was that I got all fields in one column. Troubleshooting that now.

Could that be related to https://github.com/cloudyr/limer/commit/d7c0c3ecf4e2393045711622589165e75c3b5f4d ?

[r0berts]

@Jan-E That is right - it was. It is because I had an older version (Version 2.67.1+170626) where the field separator was different. I solved this by installing a version pre-29th April. install_github("cloudyr/limer", ref = "f0e2acced9178133f56f09f40ba690d6787eda64") - I will update the installation some time soon, but presently I just needed the exporting to work.

[Jan-E]

@r0berts We had it the other way around. Limesurvey 3.12.1, but a limer that was too old. So we just had to update limer. I will see if I can patch it do do a version-specific base64_to_df.

[Jan-E]

The PR seems to be already there: https://github.com/cloudyr/limer/pull/42

[r0berts]

@Jan-E this is good, perhaps to have an option to supply version or other switch. If need be, I could do some work on documenting limer, so that it would be easier for newcomers to figure this out. I could do something like a manpage from a begginner's viewpoint, if you'd like - maybe to supplement the readme file?

[Jan-E]

I am already experimenting to get the version number:

get_session_key()
lsversion<-call_limer(method="get_site_settings",params=list(sSetttingName="versionnumber"))
lsmajor<-substr(lsversion, 1, 1)
print(lsmajor)

In my case lsversion becomes "3.12.1" and lsmajor "3". With this it should be possible to restrict https://github.com/cloudyr/limer/commit/d7c0c3ecf4e2393045711622589165e75c3b5f4d to versions > 2.

[Jan-E]

@r0berts Could you test devtools::install_github("toolsforresearch/limer", username = NULL, ref = "ls_v2_export_responses")

https://github.com/cloudyr/limer/commit/3b1d2d12799815e02a66818c6c93473db550a999 handles LS v2 the old way and LS v3 with a sep = ";". I tested it for Limesurvey 3.12.1.

Can you confirm it works in LS v2? Then I will create e new Pull Request.

[r0berts]

@Jan-E It seems NOT to work. There are error messages:

Error in if (lsmajor < 3) { : missing value where TRUE/FALSE needed
In addition: Warning message:
In base64_to_df(unlist(results)) : NAs introduced by coercion

And no data are received. When I quit the session (restart rstudio) and run (knitr) the report it does not work at library loading stage:

Quitting from lines 13-50 (surveyProc.rmd) 
Error in if (lsmajor < 3) { : missing value where TRUE/FALSE needed
Calls: <Anonymous> ... withVisible -> eval -> eval -> get_responses -> base64_to_df
In addition: Warning message:
In block_exec(params) : failed to tidy R code in chunk <setup>
reason: Error in loadNamespace(name) : there is no package called 'formatR'

But when I load libraries manually I do not seem to get that error on loading, but later no data received with the first error.

[Jan-E]

Looks like lsmajor is not set. Could you run this in a R prompt, after entering your lime_api options:

library(limer)
get_session_key()
lsversion<-call_limer(method="get_site_settings",params=list(sSetttingName="versionnumber"))
print(lsversion)
lsmajor<-as.numeric(substr(lsversion, 1, 1))
print(lsmajor)

[r0berts]

@Jan-E Yes, lsmajor is returned as list with this content:

> str(lsversion)
List of 1
 $ status: chr "Invalid setting"

Maybe it is due to this - from API reference: Get a global setting get_site_settings(string $sSessionKey, string $sSetttingName) : string|array Function to query site settings. Can only be used by super administrators. As I connect from a plain user account. - Yess that is it. If I substitute my superuser account the version string reported correctly. I am not sure RPC use plaintext or ssl in connecting to limesurvey, but I really liked to use a non-privileged account. I am giving report capability to plain users and I would not want to give s-admin password to them. Maybe a simple solution is to add an extra option like:

options(lime_api = limeAPI)
options(lime_username = "user1")
options(lime_version = "2")

It could by default be set to 3 and it could also have the automatic checking running alongside - in case it is run from super-admin account.

[Jan-E]

@r0berts Ouch. We also had a script with LS 2 and a non-superadmin RPC2 user. Hence, it failed. I now tried another method: test if 'copy_survey' exists. copy_survey was introduced in LS 3.0.

https://github.com/toolsforresearch/limer/commit/d92be6f52ad50e1dcffdca9d7af4fbe3a93931d7?diff=unified contains the final function:

base64_to_df <- function(x) {
  raw_csv <- rawToChar(base64enc::base64decode(x))

  have_copy_survey <- call_limer(method="copy_survey")
  # returns NULL in LS2
  # returns array(status => "Copy failed", error => "No survey ID has been provided. Cannot copy survey") in LS3
  if (is.null(have_copy_survey)) {
    return(read.csv(textConnection(raw_csv), stringsAsFactors = FALSE))
  } else {
    return(read.csv(textConnection(raw_csv), stringsAsFactors = FALSE, sep = ";"))
  }
}

I confirmed that this works in LS Version 2.6.4-lts with a non-superadmin account. Could you once again test the patch?

devtools::install_github("toolsforresearch/limer", username = NULL, ref = "ls_v2_export_responses_version_2")

Thanks.

[tammoterhark]

@Jan-E:

get_session_key()
lsversion<-call_limer(method="get_site_settings",params=list(sSetttingName="versionnumber"))
lsmajor<-substr(lsversion, 1, 1)
print(lsmajor)

Isn't there one "t" too many in SettingName? ;-)

[Jan-E]

Yes, but they are quite convinced, that you spell it that way https://github.com/LimeSurvey/LimeSurvey/blob/master/application/helpers/remotecontrol/remotecontrol_handle.php#L95

[r0berts]

@Jan-E

Hi Jan, sorry I was away. I tested today. Your version works fine on my 2.67 - it gets data perfectly alright with non admin account. Then I tested it on the latest version of LS (Version 3.14.1+180731) running on my laptop - it fails with this error:

 > initData <- get_responses(limeSurveyNumber)  
Error: lexical error: invalid char in json text.  
                                       <!DOCTYPE html> <html lang="en"  
                     (right here) ------

I also tested the latest limesurvey version with the version from standard repo https://github.com/cloudyr/limer and that works fine on the latest.

[tammoterhark]

@Jan-E Would it be an idea to have two versions of this piece of software: limer and limer3, to avoid this confusion.

And to persuade the LS developers to make it possible to ask the API for a version number?

Using LS3.14 I ran into this problem again.