search

natverse / neuprintr

R client utilities for interacting with the neuPrint connectome analysis service
http://natverse.org/neuprintr
3 stars 3 forks source link

Feature/adjacency speedup #122

Closed jefferis closed 4 years ago

jefferis commented 4 years ago

There turned out to be quite a lot of fat in the code that was processing the results

master:

> system.time(neuprint_get_adjacency_matrix(alids, cache=T,sparse=F))
   user  system elapsed 
  7.459   1.972  17.701 
> system.time(neuprint_get_adjacency_matrix(alids, cache=T,sparse=F))
   user  system elapsed 
  6.874   2.748   9.632 
> system.time(neuprint_get_adjacency_matrix(alids, cache=T,sparse=F))
   user  system elapsed 
  1.375   0.328   8.722 
> system.time(neuprint_get_adjacency_matrix(alids, cache=T,sparse=F))
   user  system elapsed 
  0.616   0.117   0.734

So there is 2x speedup in the first query and >10x speedup for subsequent cached queries.

jefferis commented 4 years ago

We also now have an option of sparse=TRUE to get the adjacency matrix sparse format, which is a good idea for large results.