search

dime-worldbank / googletraffic

R package to query Google Maps traffic data and transform into data
Other
58 stars 9 forks source link

Package does no longer seem to work #7

Open Laurent-Smeets-GSS-Account opened 1 year ago

Laurent-Smeets-GSS-Account commented 1 year ago

Code that used to run just fine, does not work any more 

r <- gt_make_raster(location   = c(40.712778, -74.006111),
                    height     = 2000,
                    width      = 2000,
                    zoom       = 16,
                    google_key = google_key)

r

returns:

class      : RasterLayer 
dimensions : 2010, 2006, 4032060  (nrow, ncol, ncell)
resolution : 2.15e-05, 1.63e-05  (x, y)
extent     : -74.02768, -73.98455, 40.69639, 40.72915  (xmin, xmax, ymin, ymax)
crs        : +proj=longlat +datum=WGS84 +no_defs 
source     : memory
names      : layer 
values     : NA, NA  (min, max)

and 

## Plot
r_df <- rasterToPoints(r, spatial = TRUE) %>% as.data.frame()

names(r_df) <- c("value", "x", "y")

r_df
[1] value x     y    
<0 rows> (or 0-length row.names)

could it be that the Google API has changed and that scraping requests like this are blocked?

RCura commented 1 year ago

Had the same problem for ~1 week, I just fixed it by replacing the calls to the webshot package by calls to webshot2, and it now works for me. Maybe check my fork there : https://github.com/RCura/googletraffic

ramarty commented 1 year ago

Thanks @RCura! Just accepted the pull request, so the development version should now work:

devtools::install_github("dime-worldbank/googletraffic")

I'll get this on CRAN asap.

afeefpulimala commented 1 year ago

Please let me know when you update the code. I am using this to collect data for my project

afeefpulimala commented 1 year ago

It shows the error

image
ramarty commented 1 year ago

@afeefpulimala New version on CRAN now. Is it working for you, or are you still running into issues?

afeefpulimala commented 1 year ago

I am still having the issue that I posted above

ramarty commented 1 year ago

@afeefpulimala After you run the code and get the error, could you run sessionInfo() and copy/paste the output here?

afeefpulimala commented 1 year ago

R version 4.2.2 (2022-10-31 ucrt) Platform: x86_64-w64-mingw32/x64 (64-bit) Running under: Windows 10 x64 (build 25295)

Matrix products: default

locale: [1] LC_COLLATE=English_India.utf8 LC_CTYPE=English_India.utf8
[3] LC_MONETARY=English_India.utf8 LC_NUMERIC=C
[5] LC_TIME=English_India.utf8

attached base packages: [1] stats graphics grDevices utils datasets methods base

other attached packages: [1] raster_3.6-14 sp_1.6-0 dplyr_1.1.0 googletraffic_0.1.2

loaded via a namespace (and not attached): [1] Rcpp_1.0.10 lattice_0.20-45 png_0.1-8 class_7.3-20
[5] ps_1.7.2 digest_0.6.31 utf8_1.2.3 mime_0.12
[9] R6_2.5.1 tiff_0.1-11 webshot2_0.1.0 evaluate_0.20
[13] e1071_1.7-13 pillar_1.8.1 rlang_1.0.6 curl_5.0.0
[17] Matrix_1.5-1 rmarkdown_2.20 apcluster_1.4.10 stringr_1.5.0
[21] htmlwidgets_1.6.1 proxy_0.4-27 shiny_1.7.4 xfun_0.37
[25] compiler_4.2.2 httpuv_1.6.9 pkgconfig_2.0.3 htmltools_0.5.4
[29] websocket_1.4.1 tidyselect_1.2.0 tibble_3.1.8 ColorNameR_0.1.0
[33] codetools_0.2-18 fansi_1.0.4 later_1.3.0 sf_1.0-9
[37] grid_4.2.2 jsonlite_1.8.4 xtable_1.8-4 lifecycle_1.0.3
[41] DBI_1.1.3 magrittr_2.0.3 units_0.8-1 KernSmooth_2.23-20 [45] cli_3.6.0 stringi_1.7.12 promises_1.2.0.1 googleway_2.7.6
[49] plotwidgets_0.5.1 ellipsis_0.3.2 generics_0.1.3 vctrs_0.5.2
[53] schemr_0.2.0 tools_4.2.2 glue_1.6.2 purrr_1.0.1
[57] jpeg_0.1-10 processx_3.8.0 fastmap_1.1.1 yaml_2.3.7
[61] chromote_0.1.1 terra_1.7-3 classInt_0.4-9 OpenImageR_1.2.8
[65] knitr_1.42

sbarzin commented 1 year ago

I have the same issue as @Laurent-Smeets-GSS-Account, I just get returned an empty element! (I have updated to the new CRAN version already)

r <- googletraffic::gt_make_raster(location = c(40.712778, -74.006111),

ramarty commented 1 year ago

Hi @sbarzin: Could you run sessionInfo() and copy/paste what you see there?

ramarty commented 1 year ago

@sbarzin while I troubleshoot this, one thing to also potentially check is to see if you get data by increasing the webshot delay time -- as sometimes the google traffic data can take some time to load, and if it doesn't load you may just see all NAs. Maybe try something like this? (The new CRAN release uses a new version of webshot, so may need to adjust the delay time for that... but anyways, just a quick thought and am working on troubleshooting!)

r <- googletraffic::gt_make_raster(location = c(40.712778, -74.006111),
                                   height     = 2000,
                                   width      = 2000,
                                   zoom       = 16,
                                   webshot_delay = 30,
                                   google_key = google_key)
sbarzin commented 1 year ago

@ramarty thanks! The webshot_delay worked perfectly, and I now have results with values!

aprilianidiar commented 1 year ago

I've got the same issue like @afeefpulimala, although I have tried using webshot_delay or updating the package from CRAN. I can see the data from HTML format but it was unable to make the raster layer. Any insight on this kind of issue?

Laurent-Smeets-GSS-Account commented 1 year ago

Some of the problems I had seem to have gone or at least changed, but I now get another type of error.

If I try to run 

googletraffic::gt_make_png(location   = c(40.712778, -74.006111),
                           height     = 2000,
                           width      = 2000,
                           zoom       = 16,
                           webshot_delay = 30,
                           out_filename = "test.png",
                           google_key = google_key,
                           )

I am getting this error message

Pausing for 30 seconds to allow traffic data to render
file://C:\Users\laure\AppData\Local\Temp\Rtmp4gmfwF\file4e345e646bd.html screenshot completed
Error in png::readPNG(file.path(filename_dir, paste0(filename_only, ".png"))) : 
  unable to open C:\Users\laure\AppData\Local\Temp\Rtmp4gmfwF\file4e345e646bd/file4e345e646bd.png
In addition: Warning messages:
1: In file(con, "wb") :
  cannot open file 'C:\Users\laure\AppData\Local\Temp\Rtmp4gmfwF\file4e345e646bd/file4e345e646bd.png': No such file or directory
2: In onRejected(...) :
  An error occurred: Error in file(con, "wb"): cannot open the connection

3: In normalizePath(path.expand(path), winslash, mustWork) :
  path[1]="An error occurred: Error in file(con, "wb"): cannot open the connection
": The filename, directory name, or volume label syntax is incorrect

This is my session info

> sessionInfo()
R version 4.2.1 (2022-06-23 ucrt)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 22621)

Matrix products: default

locale:
[1] LC_COLLATE=English_United Kingdom.utf8  LC_CTYPE=English_United Kingdom.utf8    LC_MONETARY=English_United Kingdom.utf8
[4] LC_NUMERIC=C                            LC_TIME=English_United Kingdom.utf8    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] raster_3.6-14       sp_1.6-0            dplyr_1.1.0         ggplot2_3.4.1       googletraffic_0.1.2

loaded via a namespace (and not attached):
 [1] pkgload_1.3.2      jsonlite_1.8.4     shiny_1.7.4        tiff_0.1-11        yaml_2.3.7         remotes_2.4.2      sessioninfo_1.2.2 
 [8] pillar_1.8.1       lattice_0.20-45    glue_1.6.2         googleway_2.7.6    digest_0.6.29      promises_1.2.0.1   colorspace_2.1-0  
[15] websocket_1.4.1    htmltools_0.5.4    httpuv_1.6.8       Matrix_1.4-1       webshot2_0.1.0     pkgconfig_2.0.3    devtools_2.4.5    
[22] purrr_1.0.1        xtable_1.8-4       scales_1.2.1       processx_3.8.0     plotwidgets_0.5.1  terra_1.7-3        jpeg_0.1-10       
[29] later_1.3.0        OpenImageR_1.2.8   tibble_3.1.8       proxy_0.4-27       generics_0.1.3     usethis_2.1.6      ellipsis_0.3.2    
[36] cachem_1.0.6       withr_2.5.0        cli_3.4.1          magrittr_2.0.3     crayon_1.5.2       mime_0.12          evaluate_0.20     
[43] memoise_2.0.1      ps_1.7.2           apcluster_1.4.10   fs_1.6.0           fansi_1.0.4        class_7.3-20       pkgbuild_1.4.0    
[50] profvis_0.3.7      tools_4.2.1        prettyunits_1.1.1  lifecycle_1.0.3    stringr_1.5.0      munsell_0.5.0      callr_3.7.3       
[57] compiler_4.2.1     e1071_1.7-12       chromote_0.1.1     rlang_1.0.6        classInt_0.4-8     units_0.8-1        grid_4.2.1        
[64] schemr_0.2.0       rstudioapi_0.14    htmlwidgets_1.6.1  miniUI_0.1.1.1     rmarkdown_2.20     gtable_0.3.1       codetools_0.2-18  
[71] DBI_1.1.3          curl_5.0.0         R6_2.5.1           knitr_1.42         fastmap_1.1.0      utf8_1.2.2         rprojroot_2.0.3   
[78] KernSmooth_2.23-20 desc_1.4.2         stringi_1.7.12     Rcpp_1.0.10        vctrs_0.5.2        sf_1.0-8           ColorNameR_0.1.0  
[85] png_0.1-8          xfun_0.36          tidyselect_1.2.0   urlchecker_1.0.1
gisynw commented 1 year ago

@afeefpulimala , did you use the windows system? I have the same issue in the window system, when I changed to Mac, this issue does not appear again. I thought it is because of the tempfile() in the window system, hope someone can come out with a solution for this issue. Thanks!

ramarty commented 1 year ago

@afeefpulimala @GISYNW @Laurent-Smeets-GSS-Account Sorry about the issue! Yep, it seems to be something specific to windows.

I just pushed an update. If you try installing the devtools version, do things then work correctly?

devtools::install_github("dime-worldbank/googletraffic")

(Update should be available on CRAN soon, but can further troubleshoot if still not working!)

Laurent-Smeets-GSS-Account commented 1 year ago

Hi @ramarty, sorry for the late reply. I am still getting the same error when using googletraffic::gt_make_png 

Pausing for 30 seconds to allow traffic data to render
file://C:\Users\laure\AppData\Local\Temp\RtmpopCzvX\file5188273866b4.html screenshot completed
Error in png::readPNG(file.path(filename_dir, paste0(filename_only, ".png"))) : 
  unable to open C:\Users\laure\AppData\Local\Temp\RtmpopCzvX\file5188273866b4/file5188273866b4.png
In addition: Warning messages:
1: In file(con, "wb") :
  cannot open file 'C:\Users\laure\AppData\Local\Temp\RtmpopCzvX\file5188273866b4/file5188273866b4.png': No such file or directory
2: In onRejected(...) :
  An error occurred: Error in file(con, "wb"): cannot open the connection

3: In normalizePath(path.expand(path), winslash, mustWork) :
  path[1]="An error occurred: Error in file(con, "wb"): cannot open the connection
": The filename, directory name, or volume label syntax is incorrect

however using gt_make_raster() directly does work. I would like to save the pngs for later analyses though. 

> sessionInfo()
R version 4.2.1 (2022-06-23 ucrt)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 22621)

Matrix products: default

locale:
[1] LC_COLLATE=English_United Kingdom.utf8 
[2] LC_CTYPE=English_United Kingdom.utf8   
[3] LC_MONETARY=English_United Kingdom.utf8
[4] LC_NUMERIC=C                           
[5] LC_TIME=English_United Kingdom.utf8    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] raster_3.6-20 sp_1.6-0      dplyr_1.1.1   ggplot2_3.4.2

loaded via a namespace (and not attached):
 [1] fs_1.6.1            OpenImageR_1.2.8    sf_1.0-12          
 [4] usethis_2.1.6       devtools_2.4.5      rprojroot_2.0.3    
 [7] tools_4.2.1         profvis_0.3.7       utf8_1.2.3         
[10] R6_2.5.1            KernSmooth_2.23-20  DBI_1.1.3          
[13] colorspace_2.1-0    urlchecker_1.0.1    withr_2.5.0        
[16] tidyselect_1.2.0    prettyunits_1.1.1   plotwidgets_0.5.1  
[19] processx_3.8.0      curl_5.0.0          compiler_4.2.1     
[22] cli_3.6.1           shinyTree_0.2.7     shinyjs_2.1.0      
[25] desc_1.4.2          ColorNameR_0.1.0    scales_1.2.1       
[28] classInt_0.4-9      callr_3.7.3         proxy_0.4-27       
[31] stringr_1.5.0       apcluster_1.4.10    digest_0.6.31      
[34] tiff_0.1-11         rmarkdown_2.21      googletraffic_0.1.2
[37] jpeg_0.1-10         pkgconfig_2.0.3     htmltools_0.5.5    
[40] sessioninfo_1.2.2   fastmap_1.1.1       htmlwidgets_1.6.2  
[43] rlang_1.1.0         rstudioapi_0.14     shiny_1.7.4        
[46] generics_0.1.3      jsonlite_1.8.4      magrittr_2.0.3     
[49] Matrix_1.5-4        Rcpp_1.0.10         munsell_0.5.0      
[52] fansi_1.0.4         lifecycle_1.0.3     terra_1.7-23       
[55] stringi_1.7.12      yaml_2.3.7          pkgbuild_1.4.0     
[58] grid_4.2.1          promises_1.2.0.1    crayon_1.5.2       
[61] miniUI_0.1.1.1      lattice_0.20-45     chromote_0.1.1     
[64] hms_1.1.3           knitr_1.42          ps_1.7.4           
[67] googleway_2.7.7     pillar_1.9.0        schemr_0.2.0       
[70] codetools_0.2-18    pkgload_1.3.2       glue_1.6.2         
[73] evaluate_0.20       data.table_1.14.8   remotes_2.4.2      
[76] png_0.1-8           vctrs_0.6.1         httpuv_1.6.9       
[79] gtable_0.3.3        purrr_1.0.1         webshot2_0.1.0     
[82] cachem_1.0.7        xfun_0.38           mime_0.12          
[85] xtable_1.8-4        e1071_1.7-13        later_1.3.0        
[88] class_7.3-20        tibble_3.2.1        shinythemes_1.2.0  
[91] websocket_1.4.1     memoise_2.0.1       units_0.8-1        
[94] ellipsis_0.3.2   
> packageVersion("googletraffic")
[1] ‘0.1.2’
waleedshkt commented 1 year ago

@ramarty

You're running on Windows OS. There's an issue with the tempfile() method specific to Windows as pointed out by @gisynw

Temp solution: try run R on Linux or MacOS

ramarty commented 8 months ago

@afeefpulimala @gisynw @Laurent-Smeets-GSS-Account New version on CRAN fixed some issues with the package working on windows -- but let me know if issues are still persisting and I'll continue troubleshooting here