persistent sessions load test added to the workflow

[kami619]

Fixes #785

• [x] Updated the Multi-AZ Create workflow to suit our current needs
• [x] Trimmed the ROSA Scaling Benchmark workflow to only perform dataset creation and benchmark and reporting
• [x] Updated the scheduled workflow to accommodate two deployments and benchmarks
• [x] Added filterable fields to Horreum report based on the deployment type (Additional work would be needed here to filter the reports in Horreum)
• [x] Local tests

Note: this will allow the test results from persistent sessions to goto horreum now, but with some additional work @andyuk1986 confirmed we would be able to filter them out based on the deployment context we are updating the upcoming horreum report schema changes.

[kami619]

Test Runs:

• Deploy job : https://github.com/kami619/keycloak-benchmark/actions/runs/8940986385
• Benchmark job : https://github.com/kami619/keycloak-benchmark/actions/runs/8941255089

Offline Sessions tables populated in the Database:

[kami619]

Interesting Observation that I found during the test run, is there is a query which runs to find out the role mappings brining in more than 7X than the regular offline sessions writes to the DB. And that seems to be growing with the number of total sessions.

cc: @ahus1

[ahus1]

@kami619 - thank you for running these tests. Let's have a call later today to dig into this.

A first look shows me that the statement you marked above is a SELECT, which is just reading from the database. As you're testing with more users that don't all fit into the cache, it is expected that during log-in Keycloak will fetch them from the database, including their roles. While this is ~92 calls per second, there is more than one role to each user which leads to ~832 rows per second retrieved from the database.

SELECTs are reading, and are then a lot faster than all writes which must send data to the other instances in the other AZ. We might also get into a situation were we saturate the CPU on our DB instance. It would be good to have a look at that as well.

[kami619]

@ahus1 sure thing, I misspoke it as writes instead of rows in my previous comment. Yup, its brining in lot more rows, would be interesting to see the load on CPU when this query runs.

[kami619]

@andyuk1986 thanks for the review and providing with the fixes for the dependency issues that you uncovered.