grunwaldlab / quiltr

R package for making website representations of directories
http://grunwaldlab.github.io/quiltr/
5 stars 1 forks source link

Look for ways to speed up `quilt` option access #57

Open zachary-foster opened 9 years ago

zachary-foster commented 9 years ago

Currently, quilt can be quite slow on large directories; it did not used to be. I think this is because of the new way options values are inferred from config files. Currently, each time a path-specific option is referenced in the code, all the config files in that path are read. It is memoized, but is still slow. Perhaps all of the config files could be read before hand and a data structure containing the option values for each path saved in a function to access option values.

zachary-foster commented 9 years ago

I thought of a way to do this:

  1. Make a two dimensional list of every file path under the specified path(s) X every quilt option.
  2. Initialize the list with the default option values
  3. find all configuration files and sort them by file hierarchy depth from path
  4. Apply the effects of each configuration file in order
  5. Apply effects of any options specified in the function call as if they were specified in a configuration file at each path
  6. store the result in a function via a closure that takes the name of an option and one or more paths.

The closure function can then be used to access the path-specific values of options in the rest of the functions.