cloudyr / limer

A LimeSurvey R Client
MIT License
66 stars 37 forks source link

Example code throws "Error: lexical error: invalid char in json text." #69

Open ulyngs opened 4 months ago

ulyngs commented 4 months ago

Hi there,

I'm trying to simply run the 'get_summary' example from the readme, and it throws this error:

call_limer(method = "get_summary", 
           params = list(iSurveyID = 919631, #one of my surveys
                         sStatname = "completed_responses"))

Error: lexical error: invalid char in json text. <!DOCTYPE html> <html lang="en" (right here) ------^

Here's the traceback

Error: lexical error: invalid char in json text.
                                       <!DOCTYPE html> <html lang="en"
                     (right here) ------^
5. parse_string(txt, bigint_as_char)
4. parseJSON(txt, bigint_as_char)
3. parse_and_simplify(txt = txt, simplifyVector = simplifyVector,
simplifyDataFrame = simplifyDataFrame, simplifyMatrix = simplifyMatrix,
flatten = flatten, ...)
2. jsonlite::fromJSON(httr::content(r, as = "text", encoding = "utf-8"))
1. call_limer(method = "export_responses", params = list(iSurveyID = 919631))

call_limer(method = "list_surveys") works, but I get this error consistently when trying to export data with an arbitrary API call.

benediktclaus commented 2 months ago

The package is dependent on httr in the background, which was superseded. It may be worth to write your own API call using the httr2 package. Something like this works fine for me:

library(httr2)

lim_url <- "your-url"
lim_user <- "your-username"
lim_pw <- "your-password"

# Retrieve session key first and then

request(lim_url) |> 
        req_body_json(
            list(
                method = "your-method",
                params = <your-parameters-as-a-list>,
                id = " "
            )
        ) |> 
        req_perform() |> 
        resp_body_json()

You could also retrieve the session key with this method by running:

request(Sys.getenv(lim_url)) |> 
    req_body_json(
        list(
            method = "get_session_key",
            params = list(
                username = lim_username,
                password = lim_password
            ),
            id = " "
        )
    ) |> 
    req_perform() |> 
    resp_body_json()
ulyngs commented 2 months ago

Immensely helpful, thank you so much!! This solved the issues for me, I now think I understand how to directly do arbitrary API calls from R:

I first get a session key:

# get session key
session_key_response <- request(credentials_limesurvey$lime_api) |> 
    req_body_json(
        list(
            method = "get_session_key",
            params = list(
                username = credentials_limesurvey$lime_username,
                password = credentials_limesurvey$lime_password
            ),
            id = " "
        )
    ) |> 
    req_perform() |> 
    resp_body_json()

Then I use this session key in subsequent API calls:

# extract the session key value
my_session_key <- session_key_response$result

# use the session key to call a method, here 'list_surveys'
request(credentials_limesurvey$lime_api) |> 
    req_body_json(
        list(
            method = "list_surveys",
            params = list(
                sSessionKey = my_session_key
            ),
            id = " "
        )
    ) |> 
    req_perform() |> 
    resp_body_json()   

Getting responses is a little fiddly because results are returned as a base 64 encoded string:

# use the session key to call the 'export_responses' method -- the parameters are documented at https://api.limesurvey.org/classes/remotecontrol-handle.html#method_export_responses
some_responses <- request(credentials_limesurvey$lime_api) |> 
    req_body_json(
        list(
            method = "export_responses",
            params = list(
                sSessionKey = my_session_key,
                iSurveyID = 946885,  #the id of my surveys
                sDocumentType = "csv"  #we want responses in CSV format
            ),
            id = " "
        )
    ) |> 
    req_perform() |> 
    resp_body_json()

# the result is a base64 encoded string, so we need to decode it
decoded_data <- base64enc::base64decode(some_responses$result)

# save out the result
writeBin(decoded_data, "decoded_responses.csv")

# read it into R --- note that the delimiter is semicolon!
responses <- read.csv("decoded_responses.csv", sep = ";")