Downloading fails: Online file links not extracted

[steffilazerte]

Recently I've run into a problem when trying to update the cache. I had some problems with the cache, so I just deleted it and decided to download from scratch, but I run into an error before the download even starts.

I've traced the problem back to get_databc_metadata() around here (and just below). Apparently, the links to the files aren't extracted as they should be. I'm not sure if this is a problem with the website (did they update the html?), my system (ubuntu/RStudio/R v3.4.4), or my package versions (I did update a bunch recently, see below). I'd be happy to supply a fix, but I'm also not certain what the output of this function should be, so I don't want to mess around too much. Does this affect any one else? Let me know if I can test anything.

Session info -------------------------------------------------------------------------------------------------
 setting  value                       
 version  R version 3.4.4 (2018-03-15)
 system   x86_64, linux-gnu           
 ui       RStudio (1.0.143)           
 language en_CA:en                    
 collate  en_CA.UTF-8                 
 tz       America/Winnipeg            
 date     2018-04-23                  

Packages -----------------------------------------------------------------------------------------------------
 package    * version    date       source                             
 assertthat   0.2.0      2017-04-11 CRAN (R 3.4.0)                     
 base       * 3.4.4      2018-03-16 local                              
 bindr        0.1        2016-11-13 CRAN (R 3.4.0)                     
 bindrcpp   * 0.2        2017-06-17 cran (@0.2)                        
 bit          1.1-12     2014-04-09 cran (@1.1-12)                     
 bit64        0.9-7      2017-05-08 cran (@0.9-7)                      
 blob         1.1.1      2018-03-25 cran (@1.1.1)                      
 broom        0.4.2      2017-02-13 CRAN (R 3.4.0)                     
 cellranger   1.1.0      2016-07-27 CRAN (R 3.4.0)                     
 cli          1.0.0      2017-11-05 cran (@1.0.0)                      
 colorspace   1.3-2      2016-12-14 CRAN (R 3.4.0)                     
 commonmark   1.2        2017-03-01 CRAN (R 3.4.0)                     
 compiler     3.4.4      2018-03-16 local                              
 crayon       1.3.4      2018-02-16 Github (gaborcsardi/crayon@95b3eae)
 curl         3.1        2017-12-12 cran (@3.1)                        
 datasets   * 3.4.4      2018-03-16 local                              
 DBI          0.8        2018-03-02 cran (@0.8)                        
 dbplyr       1.2.1      2018-02-19 cran (@1.2.1)                      
 devtools     1.13.4     2017-11-09 cran (@1.13.4)                     
 digest       0.6.15     2018-01-28 cran (@0.6.15)                     
 dplyr      * 0.7.4      2017-09-28 CRAN (R 3.4.1)                     
 forcats    * 0.2.0      2017-01-23 CRAN (R 3.4.0)                     
 foreign      0.8-69     2017-06-21 CRAN (R 3.4.0)                     
 ggplot2    * 2.2.1      2016-12-30 CRAN (R 3.4.3)                     
 glue         1.2.0      2017-10-29 cran (@1.2.0)                      
 graphics   * 3.4.4      2018-03-16 local                              
 grDevices  * 3.4.4      2018-03-16 local                              
 grid         3.4.4      2018-03-16 local                              
 gtable       0.2.0      2016-02-26 CRAN (R 3.4.0)                     
 haven        1.1.1      2018-01-18 CRAN (R 3.4.3)                     
 hms          0.4.0      2017-11-23 cran (@0.4.0)                      
 httr         1.3.1      2017-08-20 cran (@1.3.1)                      
 jsonlite     1.5        2017-06-01 cran (@1.5)                        
 lattice      0.20-35    2017-03-25 CRAN (R 3.3.3)                     
 lazyeval     0.2.1      2017-10-29 cran (@0.2.1)                      
 lubridate    1.7.4      2018-04-11 cran (@1.7.4)                      
 magrittr     1.5        2014-11-22 CRAN (R 3.4.0)                     
 memoise      1.1.0      2017-04-21 CRAN (R 3.4.0)                     
 methods    * 3.4.4      2018-03-16 local                              
 mnormt       1.5-5      2016-10-15 CRAN (R 3.4.0)                     
 modelr       0.1.1      2017-07-24 CRAN (R 3.4.4)                     
 munsell      0.4.3      2016-02-13 CRAN (R 3.4.0)                     
 nlme         3.1-131    2017-02-06 CRAN (R 3.4.0)                     
 parallel     3.4.4      2018-03-16 local                              
 pillar       1.1.0      2018-01-14 cran (@1.1.0)                      
 pkgconfig    2.0.1      2017-03-21 CRAN (R 3.4.0)                     
 plyr         1.8.4      2016-06-08 CRAN (R 3.4.0)                     
 psych        1.7.5      2017-05-03 CRAN (R 3.4.0)                     
 purrr      * 0.2.4      2017-10-18 cran (@0.2.4)                      
 R6           2.2.2      2017-06-17 cran (@2.2.2)                      
 rappdirs     0.3.1      2016-03-28 CRAN (R 3.4.0)                     
 Rcpp         0.12.16    2018-03-13 cran (@0.12.16)                    
 readr      * 1.1.1      2017-05-16 cran (@1.1.1)                      
 readxl       1.0.0      2017-04-18 CRAN (R 3.4.0)                     
 rems       * 0.4.0.9999 <NA>       local                              
 reshape2     1.4.3      2017-12-11 cran (@1.4.3)                      
 rlang        0.2.0      2018-02-20 CRAN (R 3.4.4)                     
 roxygen2     6.0.1      2017-02-06 CRAN (R 3.4.0)                     
 RSQLite      2.1.0      2018-03-29 cran (@2.1.0)                      
 rstudioapi   0.7        2017-09-07 cran (@0.7)                        
 rvest        0.3.2      2016-06-17 CRAN (R 3.4.0)                     
 scales       0.5.0.9000 2017-12-14 Github (hadley/scales@d767915)     
 stats      * 3.4.4      2018-03-16 local                              
 storr        1.1.3      2017-12-15 cran (@1.1.3)                      
 stringi      1.1.7      2018-03-12 cran (@1.1.7)                      
 stringr    * 1.3.0      2018-02-19 cran (@1.3.0)                      
 testthat     2.0.0      2017-12-13 cran (@2.0.0)                      
 tibble     * 1.4.2      2018-01-22 cran (@1.4.2)                      
 tidyr      * 0.8.0      2018-01-29 cran (@0.8.0)                      
 tidyverse  * 1.2.1      2017-11-14 CRAN (R 3.4.4)                     
 tools        3.4.4      2018-03-16 local                              
 utils      * 3.4.4      2018-03-16 local                              
 withr        2.1.1.9000 2018-01-03 Github (jimhester/withr@df18523)   
 xml2         1.2.0      2018-01-24 CRAN (R 3.4.4)

[steffilazerte]

Thinking about it a bit more, I suspect a change in the website, as a colleague brought this to my attention when suddenly they too were running into rems errors (they are on Windows and I don't think they updated anything in the meanwhile).

[ateucher]

Thanks so much @steffilazerte, and for the 🕵️‍♀️ work.

[ateucher]

@steffilazerte @j-krogh this should be fixed now - would love if you can test it out!

[j-krogh]

@ateucher Seems to be working for me. Thanks for the quick fix!

[steffilazerte]

@ateucher Same here, thanks! :rocket: