establish CI benchmarking of "fuse" operation

[yarikoptic]

• use asv following the recipe in https://github.com/datalad/datalad/ and/or https://github.com/con/fscacher/
• target dataset could be established dynamically as in fscacher or could be some tagged release among https://github.com/dandi/dandisets/ for which we must keep providing data etc for eternity
• basic benchmarks should be "cold" (ensure that is done only after the initial mount, file system underneath original repo could be "warm" already) and "warm" (2nd time - so could take advantage of some lru_cacheing we use etc)
  • ls -- pure listing of directories without needing any attributes (size, is symlink or not etc)
  • ls -l -- listing of immediate attributes but not dereferencing (e.g. for the size)
  • ls -lL -- listing with dereferencing, thus needing to parse the key etc
  • head operation ? -- not sure about this one since would be dependent on network speeds if we just go for some external dandiset as the target

[jwodder]

@yarikoptic

basic benchmarks should be "cold" ... and "warm"

asv doesn't really allow that. For one thing, based on the documentation, asv deliberately runs benchmark functions a number of times to "warm them up" before starting the actual timing, and even after that the benchmarks are run multiple times again.

[yarikoptic]

judging from https://github.com/airspeed-velocity/asv/blob/HEAD/test/benchmark/cache_examples.py#L90 benchmark could have warmup_time overridden to be 0, and number and repeat set to 1, which would give us a "cold run" sample, wouldn't it?

[jwodder]

@yarikoptic I'm not sure. I think asv does some interleaving of benchmark runs, so we could possibly end up with a "cold" and "warm" benchmark running on the same mount at the same time. I think. The documentation leaves some things to be desired.