ecohealthalliance / yenpathy

Yen's K Shortest Paths in R, Quickly
Other
7 stars 3 forks source link

reproduction of benchmarking demo #8

Open corybrunson opened 5 years ago

corybrunson commented 5 years ago

Hi @toph-allen — I'm having trouble reproducing the benchmark analysis at the end of the software paper, specifically the dramatic increase in shortest_paths() runtime with mode = "all". In case it's due to a differently sampled random graph, is there a standard or empirical graph you've demonstrated the comparison on?

library(igraph)
#> 
#> Attaching package: 'igraph'
#> The following objects are masked from 'package:stats':
#> 
#>     decompose, spectrum
#> The following object is masked from 'package:base':
#> 
#>     union
library(yenpathy)
set.seed(42)
network <- sample_smallworld(1, 20, 4, .05)
bench::mark(
  shortest_paths(network, from = 1, to = 10),
  k_shortest_paths(network, from = 1, to = 10, k = 1),
  check = FALSE
)
#> # A tibble: 2 x 6
#>   expression                                            min median
#>   <bch:expr>                                          <bch> <bch:>
#> 1 shortest_paths(network, from = 1, to = 10)          209µs  231µs
#> 2 k_shortest_paths(network, from = 1, to = 10, k = 1) 526µs  574µs
#> # … with 3 more variables: `itr/sec` <dbl>, mem_alloc <bch:byt>,
#> #   `gc/sec` <dbl>
network <- sample_smallworld(1, 8, 3, .05)
bench::mark(
  shortest_paths(network, from = 1, to = 5, mode = "all"),
  k_shortest_paths(network, from = 1, to = 5, k = 1),
  check = FALSE
)
#> # A tibble: 2 x 6
#>   expression                                                min median
#>   <bch:expr>                                              <bch> <bch:>
#> 1 shortest_paths(network, from = 1, to = 5, mode = "all") 207µs  224µs
#> 2 k_shortest_paths(network, from = 1, to = 5, k = 1)      424µs  444µs
#> # … with 3 more variables: `itr/sec` <dbl>, mem_alloc <bch:byt>,
#> #   `gc/sec` <dbl>

Created on 2019-10-01 by the reprex package (v0.2.1)

Additionally, i think it would better demonstrate your point to use k = 6 or some other non-1 number (depending on network) in the second benchmark test, since this is the use case described in the text.

Part of this JOSS review.

toph-allen commented 5 years ago

Agreed on both points — I'll amend them to use fixed graphs and to run at a higher value of k.