Certification question

[jetOak]

I've been reading the text on the wiki and I had a question about the act certification criteria for single drives. In the readme I find encountered this line:

Run ACT for 24 hrs using the 3x test (6000 reads/sec and 3000 writes/sec). The drive passes this test if less than 5% of operations fail to complete in 1 ms or less.

Later in the document I encountered these lines:

In any one-hour period of an ACT performance test, we expect that:

fewer than 5% of transactions fail to complete in 1 ms fewer than 1% of transactions fail to complete in 8 ms fewer than 0.1% of transactions fail to complete in 64 ms The max line of the output shows the highest values observed in any single slice (hour) of time and the values on the max line should not exceed the allowable error values specified above.

My question is which of the above is correct? Does the drive pass if " if less than 5% of operations fail to complete in 1 ms or less" for the total number of operations or is it per one hour time period as described in the second quote above?

Let's say I have these results from a drive:

     trans                  device

%>(ms) %>(ms)

slice 1 8 64 1 8 64

---

1 1.67 0.00 0.00 1.63 0.00 0.00 2 1.38 0.00 0.00 1.32 0.00 0.00 3 1.80 0.14 0.00 1.56 0.08 0.00 4 6.43 0.00 0.00 1.39 0.00 0.00 5 1.68 0.00 0.00 1.65 0.00 0.00 6 1.37 0.00 0.00 1.33 0.00 0.00 7 1.44 0.00 0.00 1.41 0.00 0.00 8 1.41 0.00 0.00 1.35 0.00 0.00 9 2.70 0.73 0.00 1.91 0.08 0.00 10 1.54 0.00 0.00 1.51 0.00 0.00 11 1.53 0.00 0.00 1.48 0.00 0.00 12 1.47 0.00 0.00 1.43 0.00 0.00 13 1.67 0.00 0.00 1.63 0.00 0.00 14 1.38 0.00 0.00 1.32 0.00 0.00 15 1.80 0.14 0.00 1.56 0.08 0.00 16 2.43 0.00 0.00 1.39 0.00 0.00 17 1.68 0.00 0.00 1.65 0.00 0.00 18 1.37 0.00 0.00 1.33 0.00 0.00 19 1.44 0.00 0.00 1.41 0.00 0.00 20 1.41 0.00 0.00 1.35 0.00 0.00 21 2.70 0.73 0.00 1.91 0.08 0.00 22 1.54 0.00 0.00 1.51 0.00 0.00 23 1.53 0.00 0.00 1.48 0.00 0.00 24 1.47 0.00 0.00 1.43 0.00 0.00

---

avg 1.62 0.07 0.00 1.50 0.01 0.00 max 6.43 0.73 0.00 1.91 0.08 0.00

As we can see at hour 4 the percentage of operations that failed to complete in under 1ms is greater than 5% and thus the drive failed for that time period. As it fails for that one time period then it fails certification at that workload, correct?

[Alb0t]

No idea why they haven't closed this.. But it's up to the owner to really decide what's good or not. You might want to look at that slice more closely. But - per their guidelines it failed. I'd try running it again and making sure nothing else is mudding the results up, because we would normally expect latency at the end of the test not right there.

[kportertx]

This has since been addressed: https://github.com/aerospike/act#5-evaluate-devices-by-the-standard-passfail-criteria