Speed performance estimates

mediachain / mediachain-indexer

search, dedupe, and media ingestion for mediachain

33 stars 14 forks source link

Speed performance estimates #9

Closed parkan closed 7 years ago

parkan commented 8 years ago

[ ] - Measure queries per second
[ ] - Per-shard speedup
[ ] - (Optional) Round-trip latency

Perform these over a range of image_count & shard_count combinations:

For a range of number of images [1K, 1MM, 10MM, 100MM, 1B]:
  - Throughput: Measure queries per second.
  For a range of number of ES nodes:
      - Scalability: Measure per-shard speedup based on throughput.

- Round-trip latency too?

autoencoder commented 8 years ago

Initial testing: 1.7MM artefact indexing in 12 minutes 10 seconds. Surpassing the current blockchain rate enough so that this shouldn't be the bottleneck in the near future.

autoencoder commented 7 years ago

Continuously changing and numerous dimensions of performance. Better to put these numbers in a wiki.