search

joachim-gassen / tidycovid19

{tidycovid19}: An R Package to Download, Tidy and Visualize Covid-19 Related Data
https://joachim-gassen.github.io/tidycovid19/
Other
146 stars 44 forks source link

Issue in `gtrendsR` breaking `download_google_trends_data` #29

Closed aedobbyn closed 4 years ago

aedobbyn commented 4 years ago

Hi! Thanks for the great package.

Looks to me like there's a bug in the gtrendsR package (in gtrendsR:::related_topics) that's affecting download_google_trends_data.

Might be worth wrapping all the functions that make up download_merged_data in a trycatch (maybe a purrr::possibly with otherwise set to tibble()) so that download_merged_data can download all sources that are currently working. Happy to do that and submit a PR if you like.

Let me know if you want any more info on the bug.

library(tidycovid19)

download_google_trends_data()
#> Start downloading Google Trends data
#> Pulling Google trend data for IT ...
#> Error in `[<-.data.frame`(`*tmp*`, , timevar, value = "subject"): replacement has 1 row, data has 0

packageVersion("tidycovid19")
#> [1] '0.0.0.9000'
packageVersion("gtrendsR")
#> [1] '1.4.6'

devtools::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#>  setting  value                       
#>  version  R version 4.0.0 (2020-04-24)
#>  os       macOS Catalina 10.15.5      
#>  system   x86_64, darwin17.0          
#>  ui       X11                         
#>  language (EN)                        
#>  collate  en_US.UTF-8                 
#>  ctype    en_US.UTF-8                 
#>  tz       America/New_York            
#>  date     2020-06-14                  
#> 
#> ─ Packages ───────────────────────────────────────────────────────────────────
#>  package      * version    date       lib
#>  anytime        0.3.7      2020-01-20 [1]
#>  assertthat     0.2.1      2019-03-21 [1]
#>  backports      1.1.7      2020-05-13 [1]
#>  callr          3.4.3      2020-03-28 [1]
#>  cli            2.0.2      2020-02-28 [1]
#>  colorspace     1.4-1      2019-03-18 [1]
#>  countrycode    1.2.0      2020-05-22 [1]
#>  crayon         1.3.4      2017-09-16 [1]
#>  curl           4.3        2019-12-02 [1]
#>  desc           1.2.0      2018-05-01 [1]
#>  devtools       2.3.0      2020-04-10 [1]
#>  digest         0.6.25     2020-02-23 [1]
#>  dplyr          1.0.0      2020-05-29 [1]
#>  ellipsis       0.3.1      2020-05-15 [1]
#>  evaluate       0.14       2019-05-28 [1]
#>  fansi          0.4.1      2020-01-08 [1]
#>  fastmap        1.0.1      2019-10-08 [1]
#>  fs             1.4.1      2020-04-04 [1]
#>  generics       0.0.2      2018-11-29 [1]
#>  ggplot2        3.3.1      2020-05-28 [1]
#>  ggrepel        0.8.2      2020-03-08 [1]
#>  glue           1.4.1      2020-05-13 [1]
#>  gtable         0.3.0      2019-03-25 [1]
#>  gtrendsR       1.4.6      2020-05-17 [1]
#>  highr          0.8        2019-03-20 [1]
#>  htmltools      0.4.0      2019-10-04 [1]
#>  httpuv         1.5.4      2020-06-06 [1]
#>  jsonlite       1.6.1      2020-02-02 [1]
#>  knitr          1.28       2020-02-06 [1]
#>  later          1.1.0.1    2020-06-05 [1]
#>  lifecycle      0.2.0      2020-03-06 [1]
#>  lubridate      1.7.8      2020-04-06 [1]
#>  magrittr       1.5        2014-11-22 [1]
#>  memoise        1.1.0      2017-04-21 [1]
#>  mime           0.9        2020-02-04 [1]
#>  munsell        0.5.0      2018-06-12 [1]
#>  pillar         1.4.4      2020-05-05 [1]
#>  pkgbuild       1.0.8      2020-05-07 [1]
#>  pkgconfig      2.0.3      2019-09-22 [1]
#>  pkgload        1.1.0      2020-05-29 [1]
#>  prettyunits    1.1.1      2020-01-24 [1]
#>  processx       3.4.2      2020-02-09 [1]
#>  promises       1.1.0      2019-10-04 [1]
#>  ps             1.3.3      2020-05-08 [1]
#>  purrr          0.3.4      2020-04-17 [1]
#>  R6             2.4.1      2019-11-12 [1]
#>  rclipboard     0.1.2      2019-07-02 [1]
#>  Rcpp           1.0.4.6    2020-04-09 [1]
#>  remotes        2.1.1      2020-02-15 [1]
#>  rlang          0.4.6      2020-05-02 [1]
#>  rmarkdown      2.2        2020-05-31 [1]
#>  rprojroot      1.3-2      2018-01-03 [1]
#>  scales         1.1.1      2020-05-11 [1]
#>  sessioninfo    1.1.1      2018-11-05 [1]
#>  shiny          1.4.0.2    2020-03-13 [1]
#>  shinyjs        1.1        2020-01-13 [1]
#>  shinyWidgets   0.5.3      2020-06-01 [1]
#>  stringi        1.4.6      2020-02-17 [1]
#>  stringr        1.4.0      2019-02-10 [1]
#>  testthat       2.3.2      2020-03-02 [1]
#>  tibble         3.0.1      2020-04-20 [1]
#>  tidycovid19  * 0.0.0.9000 2020-06-14 [1]
#>  tidyselect     1.1.0      2020-05-11 [1]
#>  usethis        1.6.1      2020-04-29 [1]
#>  vctrs          0.3.1      2020-06-05 [1]
#>  withr          2.2.0      2020-04-20 [1]
#>  xfun           0.14       2020-05-20 [1]
#>  xtable         1.8-4      2019-04-21 [1]
#>  yaml           2.2.1      2020-02-01 [1]
#>  source                                     
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  Github (joachim-gassen/tidycovid19@005f805)
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#>  CRAN (R 4.0.0)                             
#> 
#> [1] /Library/Frameworks/R.framework/Versions/4.0/Resources/library

Created on 2020-06-14 by the reprex package (v0.3.0)

aedobbyn commented 4 years ago

Ah sorry, I see you're already aware of that bug. I guess my trycatch suggestion still stands, though, so the main download_merged_data can still at least partially work!

joachim-gassen commented 4 years ago

Thank you for bringing up this idea!

I agree, in general it would be good to gracefully fail on failed downloads, maybe even falling back to downloading the last cached version for the specific source.

In case of the Google trends data, it might be useful to wrap the actual gtrends call in download_google_trends as this is called by country and normally fails only for some but not for all countries.

If you feel like working on this, that would be great! purrr is currently not in the dependencies lit but, hey, {tidycovid19} is super dependency heavy anyway so yet another dependency would not hurt to much I guess.

Another thing: If you are just interested in getting download_merged_data(cached = FALSE) to work again you can also try your luck with my forked version of the {gtrendsr} package (PMassicotte/gtrendsR#353). It seems to fix the issue in {gtrendsr} and works reasonably well for me.

Cheers,

Joachim

aedobbyn commented 4 years ago

Ah, your fork is a good solution, and also download_merged_data(cached = TRUE) is great. Thanks for the quick response!