mapdist reports Error - Githubissues

WickM commented 7 years ago

An Error is returned from your function mapdist in Version 2.6.. but also in 2.7 Error in *tmp*[[c(1, 1)]] : no such index at level 1 only under the circumstance that "matching was not perfect, returning what was found."

I think this line is causing the error names(tree$rows[[c(1, 1)]]) <- tree$destination_addresses

smembreno commented 7 years ago

Not sure what it is exactly. But I'm getting the same thing and I'm assuming it's because you reached the daily Query Limit

scottmmjackson commented 6 years ago

I can't reproduce this. Do you have any more information that might help?

devtools::install_github('dkahle/ggmap') first and see if that fixes your problem.

Thanks!

afleishman commented 6 years ago

I'm getting the same error.

Here's a simple reprex:

from <- as.character(98001:98100)
to <- "98102"

df_distance <- data.frame(mapdist(from = from, to = to))

bolderaft commented 6 years ago

This error seems to be related to time of day or server load issues perhaps?

bolderaft commented 6 years ago

I'm receiving this error consistently at certain times, sporadically at others, while it runs error-free at other times. I noticed it ran well on Sunday but on Monday morning it breaks frequently, which led me to speculate that it is a server load issue.

scottmmjackson commented 6 years ago

I've created a new label for this type of issue. Query limit bugs are not great but we can find a way to make ggmap a better citizen.

dpavancini commented 6 years ago

Any news on that? or some way of bypassing it? I keep getting this error in every query I make.

waynewz90 commented 6 years ago

Hmm, I seem to be getting a similar issue. I get the error message: "matching was not perfect, returning what was found". Tried devtools::install_github('dkahle/ggmap'), and checked that I haven't exceeded the daily limit. It used to work previously, returning the expected dataframe; it now returns a list. Sharing my code below:

from <- c("161 Haig Road", "161 Haig Road", "161 Haig Road", "Katong V, Singapore", "Katong V, Singapore", "Katong V, Singapore", "Parkway Parade, Singapore", "Parkway Parade, Singapore","Parkway Parade, Singapore") 
to <- c("161 Haig Road", "Katong V, Singapore", "Parkway Parade, Singapore", "161 Haig Road", "Katong V, Singapore", "Parkway Parade, Singapore", "161 Haig Road", "Katong V, Singapore", "Parkway Parade, Singapore") 
distresult <- mapdist(from=from, to=to, mode='walking')

wsrinaut commented 6 years ago

I am having the same problem. It is a distinct error from query limit reached error. I do not get the error as soon as I run the mapdist function, rather, after it has pulled some of the distances. Sometimes it will run for a couple minutes, and sometimes I will get the error immediately.

LoryA commented 6 years ago

I was also having a similar problem. My file would run with driving mode, but not with walking mode. When I filtered out unreasonably large distances/times in the driving mode, and then ran the walking mode it, it worked! I could be fooling myself by that trick though - it might still be a server timing/load issue.

scottmmjackson commented 6 years ago

Actually, this might be a dupe of #217. Would everyone having this problem please try:

devtools::install_version("rjson", version = "0.2.15")

If this fixes your issue please thumbs-up react. If it doesn't it would be great if you posted a reprex of your issue with the session info flag turned on. E.g.:

out <- reprex::reprex({
    library(ggmap)
    # what you're trying to do here
}, si = T)
cat(out, sep="\n")

You should be able to paste the result directly into github. Get reprex here: https://github.com/tidyverse/reprex

kweiss2 commented 6 years ago

Hi all, looks like this is an issue others have encountered and still may be similar to #217. I'll try to add a bit more context to see if that can point us in the right direction.

Daily Query Limit: It doesn't appear to be related to this (distQueryCheck() returns a number >>> 0).
Per-second query: Google API mentions a limit of 50 requests per second, but I've tried throttling the requests to be one every 5 seconds, and this error message still arises
Reproducibility: In iterating the mapdist() function over a dataframe using a loop, I've found that re-running a query that produced this error often produces a valid result. @scottmmjackson The reprex() example below shows a row where this error message had occurred (row 551) on the previous effort.
Changing the override_limit = option doesn't prevent the error
json version: I'm using the suggested 0.2.15 version, but still seeing this error
Variable types: Using numeric variables for origin and destination
Google: Google does appear to be changing their requests to the API for premium plan customers, not sure if that would affect the free queries https://developers.google.com/maps/documentation/directions/usage-and-billing https://cloud.google.com/maps-platform/user-guide/

Could this be a Google Maps server-side issue? Or related to the quantity of requests?

Sample R Code

# Read in file
input <- read.csv(file = "testfile.csv")

# Subset for testing
input_file <- input[1:1000, c(5:9)]

# Blank matrix to populate
driveoutput <- matrix(data = NA, nrow = nrow(input_file), ncol = 8)

# Loop through all observations in input file
for (i in 551:nrow(input_file)){

    # Extracting the origin and destination coordinates
    O <- paste0(input_file$FROM_Y[i], ",", input_file$FROM_X[i])
    D <- paste0(input_file$NEAR_Y[i], ",", input_file$NEAR_X[i])

    # Ping Google maps API 
    output <- mapdist(from = O,
                      to = D,
                      mode = "driving",
                      override_limit = TRUE))

    driveoutput[i, 1] <- as.character(input_file[i, 1]) # Location ID
    driveoutput[i, 2] <- "driving" # Mode of transit
    driveoutput[i, 3] <- output$miles # Distance
    driveoutput[i, 4] <- output$minutes # Time

    #keep track of the progress of the request...
    print(i)
}

reprex

#> Loading required package: ggplot2
#> Warning: package 'ggplot2' was built under R version 3.5.1
#> Google Maps API Terms of Service: http://developers.google.com/maps/terms.
#> Please cite ggmap if you use it: see citation("ggmap") for details.
departureTime <- as.numeric(as.POSIXct("2018-07-02 1:00:00 EST"))
mapdist(from = "34.058449,-86.82211",
        to = "34.0493,-86.8736",
        mode = "driving",
        override_limit = TRUE,
        inject = paste0("departure_time=", departureTime))
#> Source : https://maps.googleapis.com/maps/api/distancematrix/json?origins=34.058449%2C-86.82211&destinations=34.0493%2C-86.8736&mode=driving&language=en-EN&departure_time=1530507600
#>                  from               to    m    km    miles seconds minutes
#> 1 34.058449,-86.82211 34.0493,-86.8736 7279 7.279 4.523171     465    7.75
#>       hours
#> 1 0.1291667

Session Info

devtools::session_info()
#> Session info -------------------------------------------------------------
#>  setting  value                       
#>  version  R version 3.5.0 (2018-04-23)
#>  system   x86_64, mingw32             
#>  ui       RTerm                       
#>  language (EN)                        
#>  collate  English_United States.1252  
#>  tz       America/New_York            
#>  date     2018-07-05
#> Packages -----------------------------------------------------------------
#>  package     * version date       source                       
#>  assertthat    0.2.0   2017-04-11 CRAN (R 3.4.0)               
#>  backports     1.1.2   2017-12-13 CRAN (R 3.5.0)               
#>  base        * 3.5.0   2018-04-23 local                        
#>  bindr         0.1.1   2018-03-13 CRAN (R 3.4.4)               
#>  bindrcpp      0.2.2   2018-03-29 CRAN (R 3.5.0)               
#>  bitops        1.0-6   2013-08-17 CRAN (R 3.5.0)               
#>  colorspace    1.3-2   2016-12-14 CRAN (R 3.5.0)               
#>  compiler      3.5.0   2018-04-23 local                        
#>  datasets    * 3.5.0   2018-04-23 local                        
#>  devtools      1.13.6  2018-06-27 CRAN (R 3.5.1)               
#>  digest        0.6.15  2018-01-28 CRAN (R 3.5.0)               
#>  dplyr         0.7.6   2018-06-29 CRAN (R 3.5.1)               
#>  evaluate      0.10.1  2017-06-24 CRAN (R 3.4.0)               
#>  ggmap       * 2.7.900 2018-07-05 Github (dkahle/ggmap@1c2fb46)
#>  ggplot2     * 3.0.0   2018-07-03 CRAN (R 3.5.1)               
#>  glue          1.2.0   2017-10-29 CRAN (R 3.5.0)               
#>  graphics    * 3.5.0   2018-04-23 local                        
#>  grDevices   * 3.5.0   2018-04-23 local                        
#>  grid          3.5.0   2018-04-23 local                        
#>  gtable        0.2.0   2016-02-26 CRAN (R 3.4.0)               
#>  htmltools     0.3.6   2017-04-28 CRAN (R 3.5.1)               
#>  jpeg          0.1-8   2014-01-23 CRAN (R 3.5.0)               
#>  knitr         1.20    2018-02-20 CRAN (R 3.5.0)               
#>  lazyeval      0.2.1   2017-10-29 CRAN (R 3.5.0)               
#>  magrittr      1.5     2014-11-22 CRAN (R 3.4.0)               
#>  memoise       1.1.0   2017-04-21 CRAN (R 3.4.0)               
#>  methods     * 3.5.0   2018-04-23 local                        
#>  munsell       0.5.0   2018-06-12 CRAN (R 3.4.4)               
#>  pillar        1.2.3   2018-05-25 CRAN (R 3.5.0)               
#>  pkgconfig     2.0.1   2017-03-21 CRAN (R 3.4.0)               
#>  plyr          1.8.4   2016-06-08 CRAN (R 3.5.0)               
#>  png           0.1-7   2013-12-03 CRAN (R 3.5.0)               
#>  purrr         0.2.5   2018-05-29 CRAN (R 3.5.0)               
#>  R6            2.2.2   2017-06-17 CRAN (R 3.4.0)               
#>  Rcpp          0.12.17 2018-05-18 CRAN (R 3.5.0)               
#>  RgoogleMaps   1.4.2   2018-06-08 CRAN (R 3.4.4)               
#>  rjson         0.2.15  2014-11-03 url                          
#>  rlang         0.2.1   2018-05-30 CRAN (R 3.5.0)               
#>  rmarkdown     1.10    2018-06-11 CRAN (R 3.5.0)               
#>  rprojroot     1.3-2   2018-01-03 CRAN (R 3.4.3)               
#>  scales        0.5.0   2017-08-24 CRAN (R 3.5.0)               
#>  stats       * 3.5.0   2018-04-23 local                        
#>  stringi       1.2.3   2018-06-12 CRAN (R 3.5.0)               
#>  stringr       1.3.1   2018-05-10 CRAN (R 3.5.0)               
#>  tibble        1.4.2   2018-01-22 CRAN (R 3.5.0)               
#>  tidyselect    0.2.4   2018-02-26 CRAN (R 3.5.0)               
#>  tools         3.5.0   2018-04-23 local                        
#>  utils       * 3.5.0   2018-04-23 local                        
#>  withr         2.1.2   2018-03-15 CRAN (R 3.5.0)               
#>  yaml          2.1.19  2018-05-01 CRAN (R 3.5.0)

ajacob101291 commented 5 years ago

Hi all,

I did some digging and found a reason that the "matching was not perfect, returning what was found" error can be thrown:

Part of the mapdist function checks to make sure that the address entered matches the address returned by the Google API.

 response <- httr::GET(url)
        if (response$status_code != 200L) {
            warning(tryCatch(stop_for_status(response), http_400 = function(c) "HTTP 400 Bad Request", 
                http_402 = function(c) "HTTP 402 Payment Required - May indicate over Google query limit", 
                http_403 = function(c) "HTTP 403 Forbidden - Server refuses, is the API enabled?", 
                http_404 = function(c) "HTTP 404 Not Found - Server reports page not found", 
                http_414 = function(c) "HTTP 414 URI Too Long - URL query too long", 
                http_500 = function(c) "HTTP 500 Internal Server Error", 
                http_503 = function(c) "HTTP 503 Service Unavailable - Server bogged down, try later"))
            return(return_failed_mapdist(output))
        }
        tree <- httr::content(response)
        if (length(df$to) != length(tree$destination_addresses)) {
            message("Matching was not perfect, returning all.")
            names(tree$rows[[c(1, 1)]]) <- tree$destination_addresses
            output <<- "all"
        }
        else {
            names(tree$rows[[c(1, 1)]]) <- df$to
        }
        tree$rows[[c(1, 1)]]
    }

This part is supposed to check for HTTP responses as well. However, when I run the getURL function on the maps.googleapis.com url outside of the function I get the following result:

getURL("https://maps.googleapis.com/maps/api/distancematrix/json?origins=1600 Pennsylvania Ave NW, Washington, DC 20500=4 Pennsylvania Plaza, New York, NY 10001,+NC&key=xxx&mode=driving")

>[1] "{\n   \"destination_addresses\" : [],\n   \"error_message\" : \"This API project is not authorized to use this API.\",\n   \"origin_addresses\" : [],\n   \"rows\" : [],\n   \"status\" : \"REQUEST_DENIED\"\n}\n"

I received this error because I had not authorized use of the Distance API for my key within Google cloud.

This doesn't seem to cover all cases that the error is thrown, but this could be one cause.

flozanor82 commented 3 years ago

This can be solved by going to the Google Console Cloud Platform, selecting from the API dropdown menu on the left the Distance Matrix, and enabling this specific API. The cloud platform should be linked to a personal account that can be billed. However, there is a credit of $200 to perform several queries.

hryhorczuk commented 1 year ago

I had a similar error the other day and it seemed to be in response to '#' symbols in the address.

So, '1234 School Street Apt. 2' would process but 1234 School Street #2' would cause an error. It was only when I started running them 20 at a time piecemeal through my whole dataset that I noticed why some sets would work and some not. It was not a query limit issue for me. Anyway, try removing # signs from your address strings and see if that helps. It may be one possibility.

dkahle / ggmap

mapdist reports Error #166