Fenn-CS
commented
8 months ago Load Testing Results Report
Some load testing related to these was carried using https://github.com/PermanentOrg/sftp-qa and https://github.com/PermanentOrg/sftp-qa-iac
Date: 10.05.2023 Tested System: Ubuntu 22.0
Summary
The load testing session conducted ran the variety uploads test with 5 Ubuntu machines set up to upload simultaneously to the same account.
Key Findings
- CPU Usage: While the 5 machines where uploading, CPU usage consistently reached its maximum capacity. Averaging between 95%-100%.
- Denial of Service (DOS) Issues: There is noticeable lag on the "frontend" as the page takes a lot of time to load waiting for responses from the backend. Backend login requests, such as those from non-SFTP users, were observed to time out when five simultaneous SFTP users were uploading data.
- RAM Usage: RAM usage remained stable throughout the load testing, indicating that memory resources are not a limiting factor.
Recommendation
Apart from obvious CPU scaling options, the recommended number of simultaneous users for rlcone on the m4.xlarge dev backend
Conclusion
There is still significant CPU bottlenecks with heavy parallel usage of rclone. Pointers for things that could be aggravating this bottleneck include the permanent backend taskrunner.
Noted here because it's related to performance metrics
cc: @kfogel @slifty @cecilia-donnelly
We want to get baseline metrics for fastest possible performance, so we will set up an instance in AWS' us-west-2 region and run the following tests: