Performance evaluation

[benedekh]

Evaluate the performance of your storage service implementation and compose a short report (max. 2 pages). You can use the ​Enron Email data set to populate your storage service and run experiments. Measure latency and throughput for read and write operations in varying scenarios.

• different number of clients connected to the service (e.g. 1, 5, 20)
• different number of storage nodes participating in the storage service (e.g. 1, 5, 10)
• different KV server configurations (cache sizes, strategies)

Also think of evaluating the process of scaling the system up & down, i.e., how long does it take in the different scenarios to add/remove a KVServer.

Your report should contain the quantitative results (e.g., plots) and an explanation of the results.

Mirror of the dataset: http://home.sch.bme.hu/~benjo123/enron/enron-dataset.7z (~120 MB, 7zip)

[benedekh]

Related issue #104

[benedekh]

Reopened for M5. Helper project

[benedekh]

Wrong measurements' influx DB export with metadata also (LB broke down with heap size exceed error, KVServer broke down with GC overhead error):

• 1 client, 1 server, PUT
• 5 clients, 5 servers, PUT

VM configuration:

	kvserver, LB, statsd	client
RAM	4 GB	1 GB
CPU	2 cores	1 core
SSD	60 GB	30 GB
Java	1.7.0_80-b15 (Oracle)	1.8.0_121-b13 (Oracle)

• CPU (due to /cpu/procinfo): Intel Xeon CPU E5-2650L v3 @ 1.8 GHz
• OS: Ubuntu 16.04.1. LTS x64

Scenarios:

• KVServer cache parameters: 1000 LFU
• Usual processing time for more clients (2k entries read & upload): 16:04 (c3, NYC) -> 24:49 (c4, SGP) minutes
• Usual processing time for one client (2k entries read & upload): ~11 minutes -> ~18 minutes

1vs1:

• Locations:
  • C1 (FRA1)
  • S1 (FRA1)
    • node1 Write range: (-7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f80,7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f) Hash key: 9afe8a893059867d75304f32ad85625
  • LB: NYC3
  • statsd: NYC3
• Stats:
  • 1 client, 1 server, PUT
  • 1 client, 1 server, GET

1vs5:

• Locations:
  • C1 (FRA1)
  • S1 (FRA1), S2 (SFO2), S3 (NYC3), S4 (SGP1), S5 (LON1)
    • node1 Write range: (2ba8bfca79b7b5e9152494db1f584f58,7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f) Hash key: 4e2e1cc6990a14524a114a30477dd980
    • node2 Write range: (-7aa861bcc0a2fb563aae34f98887c153,-6f428e71897f38fe656fc50304dc4f48) Hash key: -6f428e71897f38fe656fc50304dc4f48
    • node3 Write range: (-6f428e71897f38fe656fc50304dc4f47,29c3ba8c287b0670df875b4877a9a4ed) Hash key: 29c3ba8c287b0670df875b4877a9a4ed
    • node4 Write range: (29c3ba8c287b0670df875b4877a9a4ee,2ba8bfca79b7b5e9152494db1f584f57) Hash key: 2ba8bfca79b7b5e9152494db1f584f57
    • node5 Write range: (-7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f80,-7aa861bcc0a2fb563aae34f98887c154) Hash key: -7aa861bcc0a2fb563aae34f98887c154
  • LB: NYC3
  • statsd: NYC3
• Stats:
  • 1 client, 5 servers, PUT
  • 1 client, 5 servers, GET

5vs1:

• Locations:
  • C1 (FRA1), C2 (SFO2), C3 (NYC3), C4 (SGP1), C5 (LON1)
  • S1 (FRA1)
    • node1 Write range: (-7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f80,7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f) Hash key: 4ffaf793c77b602f429ba1b6b033d251
  • LB: NYC3
  • statsd: NYC3
• Stats:
  • 5 clients, 1 server, PUT
  • 5 clients, 1 server, GET

5vs5:

• Locations:
  • C1 (FRA1), C2 (SFO2), C3 (NYC3), C4 (SGP1), C5 (LON1)
  • S1 (FRA1), S2 (SFO2), S3 (NYC3), S4 (SGP1), S5 (LON1)
    • node1 Write range: (-7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f80,-6607b90ce598dc9f292fe03a601ab87e) Hash key: -6607b90ce598dc9f292fe03a601ab87e
    • node2 Write range: (-44c84c97650a9a8278b2abd9e22fc4b2,-2a408151158dbf442f3dadf8f7bfe8c0) Hash key: -2a408151158dbf442f3dadf8f7bfe8c0
    • node3 Write range: (18ca88b78ff4006c5f8828b8ac6449d4,7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f7f) Hash key: 36a6851c4a04ee4ab265f5317e1dd2da
    • node4 Write range: (-2a408151158dbf442f3dadf8f7bfe8bf,18ca88b78ff4006c5f8828b8ac6449d3) Hash key: 18ca88b78ff4006c5f8828b8ac6449d3
    • node5 Write range: (-6607b90ce598dc9f292fe03a601ab87d,-44c84c97650a9a8278b2abd9e22fc4b3) Hash key: -44c84c97650a9a8278b2abd9e22fc4b3
  • LB: NYC3
  • statsd: NYC3
• Stats:
  • 5 clients, 5 servers, PUT
  • 5 clients, 5 servers, GET

[benedekh]

Measurement results are put into the presentation.