search

ballerina-platform / ballerina-central

Ballerina Central Website
Apache License 2.0
104 stars 1 forks source link

Add performance tests to identify performance degrading due to any change #211

Open anupama-pathirage opened 1 month ago

anupama-pathirage commented 1 month ago

@Dilhasha cloned issue wso2-enterprise/ballerina-registry#1325 on 2022-01-21:

Description:

Add performance tests to identify performance degrading due to any changes and the impact on APIs invoked by external parties.

Please refer to the list of API details at https://docs.google.com/spreadsheets/d/17KbKupo8VezbXwGr58u7vUG_qEqO2glgFRPbbBOooMo/edit?usp=sharing

Describe your task(s)

Related Issues (optional):

Suggested Labels (optional):

Suggested Assignees (optional):

anupama-pathirage commented 1 month ago

@Dilhasha commented on 2022-02-08:

Please refer the Threshold Calculation tab in [1] to see how the threshold values for the above API calls are calculated.

anupama-pathirage commented 1 month ago

@Dilhasha commented on 2022-02-09:

While the above workflow can be used to collect and compare the time taken against a threshold, we need to come up with the following.

  • After deploying the changes from a PR merge to dev,

  • [x] Execute API requests to Package APIs

  • [x] Compare the execution time against the last record (whether it has increased more than 10% from previous execution time)

  • [x] Trigger a notification if there is an increase in time

  • [x] Append execution time results to a csv file in GH

anupama-pathirage commented 1 month ago

@Dilhasha commented on 2022-02-15:

Main Steps:

  • Merge the changes from a PR to Dev Environment

  • Execute pipeline task for perf degradation[1]

  • Wait until the deployment is ready

  • Execute GH workflow for performance check

GH workflow[2] for performance check :

  • Run script to persist execution time [3]

  • Commit execution time record

  • Send notification for performance degradation - a Ballerina program[4] that reads and compares the last 2 entries(whether current execution has increased more than 10% from previous execution time) and sends a chat message if there is an increase

[1] https://github.com/wso2-enterprise/ballerina-registry-control-plane/blob/bd2b3ffe426416c8668d6743c7a4e46c441f3488/.azure/dev-deploy.yml#L78

[2] https://github.com/wso2-enterprise/ballerina-registry-control-plane/blob/144a5e0ccc8f285df0b145aea59670a1f06f788e/.github/workflows/notify_performance_degradation.yml

[3] https://github.com/wso2-enterprise/ballerina-registry-control-plane/blob/144a5e0ccc8f285df0b145aea59670a1f06f788e/performance-tests/persist_execution_time.sh

[4] https://github.com/wso2-enterprise/ballerina-registry-control-plane/blob/144a5e0ccc8f285df0b145aea59670a1f06f788e/performance-tests/perf_test_runner/main.bal

anupama-pathirage commented 1 month ago

@Dilhasha commented on 2022-03-14:

TODO: There are glitches in the numbers - getting false alarms due to inconsistent numbers - need to check and correct