r-lib / httr2

Make HTTP requests and process their responses. A modern reimagining of httr.
https://httr2.r-lib.org
Other
235 stars 56 forks source link

req_verbose: body_req does not return body when containing german Umlaute (utf-8 issue?) #517

Closed werkstattcodes closed 3 weeks ago

werkstattcodes commented 3 weeks ago

Hello,

wonderful package, many thanks.

Below a reprex with two API calls.

When my body_paramscontain an umlaut, req_verbosereturns for the body_req">> 17 bytes of binary data". However, it should be >> {"WP":["GRÜNE"]} If no Umlaute are contained, the response of req_verboseis as desired: >> {"WP":["NEOS"]}

I assume this is an UTF-8/text encoding issue.

Many thanks!

library(httr2)

#WITH UMLAUT
body_params <- '{"WP":["GRÜNE"]}'

httr2::request("https://www.parlament.gv.at/Filter/api/json/post") |>
    httr2::req_url_query(
      jsMode = "EVAL",
      FBEZ = "WFW_008",
      listeId = "undefined",
      # pageNumber = "1",
      # pagesize = "10",
      showAll="true",
      feldRnr = "3",
      ascDesc = "ASC",
    ) |>
    httr2::req_headers(
      accept = "*/*",
      `accept-language` = "en-US,en;q=0.9,de-AT;q=0.8,de;q=0.7,en-AT;q=0.6",
      `content-type` = "application/json",
      dnt = "1",
      origin = "https://www.parlament.gv.at",
      priority = "u=1, i"
    ) |>
    httr2::req_body_raw(body_params, "application/json") |>
    httr2::req_verbose(body_req=T, header_req=F, header_resp = F) |>
    httr2::req_perform()
#> >> 17 bytes of binary data
#> <httr2_response>
#> POST
#> https://www.parlament.gv.at/Filter/api/json/post?jsMode=EVAL&FBEZ=WFW_008&listeId=undefined&showAll=true&feldRnr=3&ascDesc=ASC
#> Status: 200 OK
#> Content-Type: application/json
#> Body: In memory (994 bytes)

#WITHOUT UMLAUT
body_params <- '{"WP":["NEOS"]}'

httr2::request("https://www.parlament.gv.at/Filter/api/json/post") |>
    httr2::req_url_query(
      jsMode = "EVAL",
      FBEZ = "WFW_008",
      listeId = "undefined",
      # pageNumber = "1",
      # pagesize = "10",
      showAll="true",
      feldRnr = "3",
      ascDesc = "ASC",
    ) |>
    httr2::req_headers(
      accept = "*/*",
      `accept-language` = "en-US,en;q=0.9,de-AT;q=0.8,de;q=0.7,en-AT;q=0.6",
      `content-type` = "application/json",
      dnt = "1",
      origin = "https://www.parlament.gv.at",
      priority = "u=1, i"
    ) |>
    httr2::req_body_raw(body_params, "application/json") |>
    httr2::req_verbose(body_req=T, header_req=F, header_resp = F) |>
    httr2::req_perform()
#> >> {"WP":["NEOS"]} 
#> <httr2_response>
#> POST
#> https://www.parlament.gv.at/Filter/api/json/post?jsMode=EVAL&FBEZ=WFW_008&listeId=undefined&showAll=true&feldRnr=3&ascDesc=ASC
#> Status: 200 OK
#> Content-Type: application/json
#> Body: In memory (994 bytes)

Created on 2024-08-23 with reprex v2.1.0

hadley commented 3 weeks ago

Duplicate of #91