search

jumpingrivers / HIBPwned

R Package 📦 for using the HaveIBeenPwned.com API :scream:
http://jumpingrivers.github.io/HIBPwned/
Other
13 stars 4 forks source link

Travis issues #34

Open stephlocke opened 5 years ago

stephlocke commented 5 years ago

test_check("HIBPwned") ── 1. Failure: account_breaches works for a single account (@test-account_breach res[[acct]] inherits from NULL not data.frame. ── 2. Error: account_breaches works for a single account (@test-account_breaches is.numeric(act$val) is not TRUE 1: expect_gt(ncol(res[[acct]]), 1) at testthat/test-account_breaches.R:13 2: expect_compare(">", act, exp) 3: stopifnot(is.numeric(act$val), length(act$val) == 1) 4: stop(msg, call. = FALSE, domain = NA) ── 3. Failure: account_breaches works for multiple accounts (@test-account_breac res[[a]] inherits from NULL not data.frame. ── 4. Error: account_breaches works for multiple accounts (@test-account_breache is.numeric(act$val) is not TRUE 1: expect_gte(ncol(res[[a]]), 1) at testthat/test-account_breaches.R:59 2: expect_compare(">=", act, exp) 3: stopifnot(is.numeric(act$val), length(act$val) == 1) 4: stop(msg, call. = FALSE, domain = NA) ── 5. Error: account_breaches can return a single data.frame (@test-account_brea subscript out of bounds 1: account_breaches("steff.sullivan@gmail.com", as_list = FALSE) at testthat/test-account_breaches.R:126

maelle commented 5 years ago

The user agent seems to be passed correctly. The request is however forbidden.

resp$status_http()
<Status code: 403>
  Message: Forbidden
  Explanation: Request forbidden -- authorization will not help
maelle commented 5 years ago

In the API docs the only reason given for a 403 error is a missing user agent. https://haveibeenpwned.com/API/v2

We passe the User Agent in a list e.g.


  list(`api-version` = "2"
       , `User-Agent` = agent)
#> Error in eval(expr, envir, enclos): object 'agent' not found

Created on 2019-01-22 by the reprex package (v0.2.1)

And then this is how we pass it to crul, as headers

 crul::HttpClient$new(url = url,
                       headers = headers)$get()
maelle commented 5 years ago

Oh but now the Travis build no longer fails?!

maelle commented 5 years ago

Locally I still get the failures. :thinking:

maelle commented 5 years ago

@stephlocke we can rule out crul being the issue

account <- "maelle.salmon@yahoo.se"
encoded <- urltools::url_encode(account)
URL <- paste0( # nolint
  "https://haveibeenpwned.com/api/v2/breachedaccount/"
  , encoded
)

headers <- list(`api-version` = "2", 
                `User-Agent` = "R package HIBPwned")
res <- crul::HttpClient$new(url = URL,
                            headers = headers)$get()
res$request_headers
#> $`Accept-Encoding`
#> [1] "gzip, deflate"
#> 
#> $Accept
#> [1] "application/json, text/xml, application/xml, */*"
#> 
#> $`api-version`
#> [1] "2"
#> 
#> $`User-Agent`
#> [1] "R package HIBPwned"

httr::GET(URL, 
         httr::user_agent("R package HIBPwned"))
#> Response [https://haveibeenpwned.com/api/v2/breachedaccount/maelle.salmon%40yahoo.se]
#>   Date: 2019-01-28 15:59
#>   Status: 403
#>   Content-Type: text/html; charset=UTF-8
#>   Size: 1.96 kB
#> <!DOCTYPE html>
#> <head>
#> <title>Request Blocked</title>
#> <meta charset="UTF-8" />
#> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
#> </head>
#> <body>
#> <h1>You have been blocked from accessing this resource on Have I Been Pw...
#> 
#> <p>This may be due to violating one or more of <a href="https://haveibee...
#> ...

Created on 2019-01-28 by the reprex package (v0.2.1)

Session info ``` r devtools::session_info() #> ─ Session info ────────────────────────────────────────────────────────── #> setting value #> version R version 3.4.4 (2018-03-15) #> os Ubuntu 18.04.1 LTS #> system x86_64, linux-gnu #> ui X11 #> language en_US #> collate en_US.UTF-8 #> ctype en_US.UTF-8 #> tz Europe/Paris #> date 2019-01-28 #> #> ─ Packages ────────────────────────────────────────────────────────────── #> package * version date lib source #> assertthat 0.2.0 2017-04-11 [1] CRAN (R 3.4.4) #> backports 1.1.3 2018-12-14 [1] CRAN (R 3.4.4) #> callr 3.1.1 2018-12-21 [1] CRAN (R 3.4.4) #> cli 1.0.1 2018-09-25 [1] CRAN (R 3.4.4) #> crayon 1.3.4 2017-09-16 [1] CRAN (R 3.4.4) #> crul 0.7.0 2019-01-04 [1] CRAN (R 3.4.4) #> curl 3.3 2019-01-10 [1] CRAN (R 3.4.4) #> desc 1.2.0 2018-05-01 [1] CRAN (R 3.4.4) #> devtools 2.0.1 2018-10-26 [1] CRAN (R 3.4.4) #> digest 0.6.18 2018-10-10 [1] CRAN (R 3.4.4) #> evaluate 0.12 2018-10-09 [1] CRAN (R 3.4.4) #> fs 1.2.6 2018-08-23 [1] CRAN (R 3.4.4) #> glue 1.3.0.9000 2019-01-22 [1] Github (tidyverse/glue@0549457) #> highr 0.7 2018-06-09 [1] CRAN (R 3.4.4) #> htmltools 0.3.6 2017-04-28 [1] CRAN (R 3.4.4) #> httpcode 0.2.0 2016-11-14 [1] CRAN (R 3.4.4) #> httr 1.4.0 2018-12-11 [1] CRAN (R 3.4.4) #> knitr 1.21 2018-12-10 [1] CRAN (R 3.4.4) #> magrittr 1.5 2014-11-22 [1] CRAN (R 3.4.4) #> memoise 1.1.0 2017-04-21 [1] CRAN (R 3.4.4) #> pkgbuild 1.0.2 2018-10-16 [1] CRAN (R 3.4.4) #> pkgload 1.0.2 2018-10-29 [1] CRAN (R 3.4.4) #> prettyunits 1.0.2 2015-07-13 [1] CRAN (R 3.4.4) #> processx 3.2.1 2018-12-05 [1] CRAN (R 3.4.4) #> ps 1.3.0 2018-12-21 [1] CRAN (R 3.4.4) #> R6 2.3.0 2018-10-04 [1] CRAN (R 3.4.4) #> Rcpp 1.0.0 2018-11-07 [1] CRAN (R 3.4.4) #> remotes 2.0.2 2018-10-30 [1] CRAN (R 3.4.4) #> rlang 0.3.1 2019-01-08 [1] CRAN (R 3.4.4) #> rmarkdown 1.11 2018-12-08 [1] CRAN (R 3.4.4) #> rprojroot 1.3-2 2018-01-03 [1] CRAN (R 3.4.4) #> sessioninfo 1.1.1 2018-11-05 [1] CRAN (R 3.4.4) #> stringi 1.2.4 2018-07-20 [1] CRAN (R 3.4.4) #> stringr 1.3.1 2018-05-10 [1] CRAN (R 3.4.4) #> testthat 2.0.1 2018-10-13 [1] CRAN (R 3.4.4) #> triebeard 0.3.0 2016-08-04 [1] CRAN (R 3.4.4) #> urltools 1.7.1 2018-08-03 [1] CRAN (R 3.4.4) #> usethis 1.4.0.9000 2019-01-03 [1] Github (r-lib/usethis@85bf30a) #> withr 2.1.2 2018-03-15 [1] CRAN (R 3.4.4) #> xfun 0.4 2018-10-23 [1] CRAN (R 3.4.4) #> yaml 2.2.0 2018-07-25 [1] CRAN (R 3.4.4) #> #> [1] /home/maelle/R/x86_64-pc-linux-gnu-library/3.4 #> [2] /usr/local/lib/R/site-library #> [3] /usr/lib/R/site-library #> [4] /usr/lib/R/library ```
maelle commented 5 years ago

Hypotheses:

maelle commented 5 years ago

Note that the build passes on Travis.

maelle commented 5 years ago

Possibly related https://github.com/michenriksen/hibp/issues/3

maelle commented 5 years ago

See also https://community.home-assistant.io/t/haveibeenpwned-stopped-working-failed-fetching-data-http-status-code-403/92680/4

maelle commented 5 years ago

In the thread above folks write that they had to contact Troy Hunt to get their IP unblocked. :thinking:

@stephlocke if CRAN's IP was blocked by our tests, it's quite bad for #35

Besides, if we contacted the API maintainer, I wonder whether we could ask whether there was a reason in all the 403 errors/IP blocking. Bad luck for us or did we do something wrong?

maelle commented 5 years ago

For info now the tests pass on Travis. :woman_shrugging: