Closed andyuk1986 closed 1 month ago
You can find a successful run on my fork here: https://github.com/andyuk1986/keycloak-benchmark/actions/runs/9267087730
So, I am afraid that the generated report data so far was only correct for memoryUsageTest, for other 2 test runs (cpuUsage and cpuUsageForGrants was incorrectly calculated).
@andyuk1986 - thank you for figuring out the problem!
To help me understand the root cause: Can you please describe it to me? Looking at the diff, the rm
seems to do the trick?
@andyuk1986 - thank you for figuring out the problem!
To help me understand the root cause: Can you please describe it to me? Looking at the diff, the
rm
seems to do the trick? @ahus1 actually there were 2 problems:
- The floating point math operation in bash sometimes were throwing error which was causing that data was completely missing for specific test. > I have fixed this by rounding the data already on prometheus level.
- The 'rm' part was the reason of devision to 0 issue. When the statistic is read from prometheus the data is stored in the file. This is done before and after the benchmark ran and is written on new line. Then the metric calculator reads the data and makes math operations to calculate that and theoretically should remove the file for next bechmark iteration as the next benchmark uses the same file name. I was deleting one of the files and missed the second, as for ispn metric calculation I am picking 2 time sub-metrics (seconds sum and count) and store them in different files for later processing. So as one of the files was not deleted, the later benchmarks were adding the new statistics to already existing file where already data was from previous benchmark run, and after that the whole calculation was going wrong.
Thank you for the explanation and the fix!
Fixes #832