Open LecCory opened 1 year ago
We used Haibun for load testing in the past. The advantage compared to request-based testing tools is it re-uses deep testing plans with simultaenous browsers so sessions and external connections (assets, services, etc) are tested with high fidelity. However, as it is currently built to launch all instances from one base, it can only reasonably launch dozens of simultaenous browsers (we tested up to 10000 requests, over a period of time). Haibun's internal executor would benefit from a redesign to suit this approach, but first step would be to analyze what points in the managed service are important to test to determine the appropriate approach.
Ideally we'd like to test a sample load of what prod collab handles on a daily basis. We'd need to look at analytics for that unless @Phanoix has the ability to provide me access to the GA dashboard for collab. Anyway, we want to test collab running on managed services with and without the use of memcached. Once we have a copy of the prod database loaded in we want to see what the response times are like with a full database on managed hardware.
PoC using @haibun/web-playwright with Azure durable functions; https://github.com/vid/haibun-azure-functions-mqtt-poc
note that it does not use mqtt. that part tbd in terms of abstraction.
to keep this as simple and useful as possible, the test instances (a docker image that receives tests to execute) can be managed separately, either manually, using k8s, or any other system. A module (#76) will be used to orchestrate the tests themselves.
blocked by #2534: Restore collab backup