search

ajdamico / asdfree

analyze survey data for free
http://asdfree.com/
GNU General Public License v3.0
612 stars 449 forks source link

Problem downloading NHANES #357

Closed LukasWallrich closed 3 years ago

LukasWallrich commented 3 years ago

Thanks for this very useful package. Unfortunately, I cannot download the nhanes data - I get the same error message on two machines, even though they can download other data (so no issue with R connecting to the web) and even though the link lodown complains about works in the browser. Not sure what is going on.

lodown("nhanes")
building catalog for nhanes

locally downloading nhanes

downloading from URL
'https://wwwn.cdc.gov/Nchs/Nhanes/1999-2000/RXQ_DRUG.xpt'
to file
'C:\Users\lukas\AppData\Local\Temp\Rtmpov7Ntu\file2348b704d6a'

download issue with
'https://wwwn.cdc.gov/Nchs/Nhanes/1999-2000/RXQ_DRUG.xpt'

download issue with
'https://wwwn.cdc.gov/Nchs/Nhanes/1999-2000/RXQ_DRUG.xpt'

download issue with
'https://wwwn.cdc.gov/Nchs/Nhanes/1999-2000/RXQ_DRUG.xpt'

R version 4.0.4 (2021-02-15)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19041)

Matrix products: default

locale:
[1] LC_COLLATE=English_United Kingdom.1252  LC_CTYPE=English_United Kingdom.1252    LC_MONETARY=English_United Kingdom.1252
[4] LC_NUMERIC=C                            LC_TIME=English_United Kingdom.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] lodown_0.1.0              devtools_2.3.2            usethis_2.0.1             hrbrthemes_0.8.0          tidyr_1.1.3              
 [6] ggalt_0.6.2               shinyWidgets_0.6.0        gt_0.2.2                  broom_0.7.5.9000          haven_2.3.1              
[11] patchwork_1.1.1           ggplot2_3.3.3             naniar_0.6.0.9000         googleCloudStorageR_0.6.0 stringr_1.4.0            
[16] purrr_0.3.4               readr_1.4.0               dplyr_1.0.5               here_1.0.1                magrittr_2.0.1           
[21] shiny_1.6.0               bslib_0.2.4              

loaded via a namespace (and not attached):
  [1] colorspace_2.0-0     selectr_0.4-2        ellipsis_0.3.1       rsconnect_0.8.16     visdat_0.5.3         rprojroot_2.0.2     
  [7] fs_1.5.0             rstudioapi_0.13      timesaveR_0.0.1.9008 listenv_0.8.0        farver_2.1.0         remotes_2.2.0       
 [13] fansi_0.4.2          xml2_1.3.2           codetools_0.2-18     R.methodsS3_1.8.1    extrafont_0.17       cachem_1.0.4        
 [19] knitr_1.31           pkgload_1.2.0        jsonlite_1.7.2       Rttf2pt1_1.3.8       R.oo_1.24.0          clipr_0.7.1         
 [25] compiler_4.0.4       httr_1.4.2           backports_1.2.1      assertthat_0.2.1     fastmap_1.1.0        lazyeval_0.2.2      
 [31] gargle_1.0.0         cli_2.3.1            later_1.1.0.1        prettyunits_1.1.1    htmltools_0.5.1.1    tools_4.0.4         
 [37] gtable_0.3.0         glue_1.4.2           maps_3.3.0           Rcpp_1.0.6           jquerylib_0.1.3      styler_1.4.0        
 [43] vctrs_0.3.6          extrafontdb_1.0      xfun_0.22            ps_1.6.0             globals_0.14.0       rvest_1.0.0         
 [49] testthat_3.0.2       mime_0.10            lifecycle_1.0.0      pacman_0.5.1         future_1.21.0        googleAuthR_1.3.1   
 [55] MASS_7.3-53.1        scales_1.1.1         hms_1.0.0            promises_1.2.0.1     parallel_4.0.4       proj4_1.0-10.1      
 [61] rematch2_2.1.2       RColorBrewer_1.1-2   yaml_2.2.1           curl_4.3             memoise_2.0.0        gdtools_0.2.3       
 [67] sass_0.3.1           stringi_1.5.3        desc_1.3.0           checkmate_2.0.0      pkgbuild_1.2.0       zip_2.1.1           
 [73] rlang_0.4.10         pkgconfig_2.0.3      systemfonts_1.0.1    bitops_1.0-6         evaluate_0.14        htmlwidgets_1.5.3   
 [79] labeling_0.4.2       processx_3.4.5       tidyselect_1.1.0     parallelly_1.24.0    plyr_1.8.6           R6_2.5.0            
 [85] generics_0.1.0       DBI_1.1.1            pillar_1.5.1         withr_2.4.1          RCurl_1.98-1.3       ash_1.0-15          
 [91] tibble_3.1.0         crayon_1.4.1         KernSmooth_2.23-18   utf8_1.2.1           plotly_4.9.3         rmarkdown_2.7       
 [97] grid_4.0.4           data.table_1.14.0    callr_3.5.1          forcats_0.5.1        digest_0.6.27        xtable_1.8-4        
[103] R.cache_0.14.0       httpuv_1.5.5         R.utils_2.10.1       openssl_1.4.3        munsell_0.5.0        viridisLite_0.3.0   
[109] sessioninfo_1.1.1    askpass_1.1         

lodown is now exiting unexpectedly.
websites that host publicly-downloadable microdata change often and sometimes those changes cause this software to break.
if the error call stack below appears to be a hiccup in your internet connection, then please verify your connectivity and retry the download.
otherwise, please open a new issue at `https://github.com/ajdamico/asdfree/issues` with the contents of this error call stack and also the output of your `sessionInfo()`.

[[1]]
lodown("nhanes")

[[2]]
withCallingHandlers(catalog <- load_fun(data_name = data_name, 
    catalog, ...), error = function(e) {
    print(sessionInfo())
    if (grepl("cannot allocate vector of size", e)) 
        message(memory_note)
    else if (grepl("parameter must be specified", e)) 
        message(parameter_note)
    else if (grepl("to install", e)) 
        message(installation_note)
    else {
        message(unknown_error_note)
        print(sys.calls())
    }
})

[[3]]
load_fun(data_name = data_name, catalog, ...)

[[4]]
cachaca(catalog[i, "full_url"], tf, mode = "wb")

[[5]]
httr_filesize(this_url, attempts, sleepsec)

[[6]]
stop(paste0("httr::HEAD( '", url, "' )\nfailed after ", 
    initial.attempts, " attempts"))

[[7]]
.handleSimpleError(function (e) 
{
    print(sessionInfo())
    if (grepl("cannot allocate vector of size", e)) 
        message(memory_note)
    else if (grepl("parameter must be specified", e)) 
        message(parameter_note)
    else if (grepl("to install", e)) 
        message(installation_note)
    else {
        message(unknown_error_note)
        print(sys.calls())
    }
}, "httr::HEAD( 'https://wwwn.cdc.gov/Nchs/Nhanes/1999-2000/RXQ_DRUG.xpt' )\nfailed after 3 attempts", 
    base::quote(httr_filesize(this_url, attempts, sleepsec)))

[[8]]
h(simpleError(msg, call))

Error in httr_filesize(this_url, attempts, sleepsec) : 
  httr::HEAD( 'https://wwwn.cdc.gov/Nchs/Nhanes/1999-2000/RXQ_DRUG.xpt' )
failed after 3 attempts
In addition: Warning messages:
1: Unknown or uninitialised column: `unzip_folder`. 
2: Unknown or uninitialised column: `output_folder`.
ajdamico commented 3 years ago

hi @LukasWallrich could you debug(lodown:::lodown_nhanes) and submit a pull request?