stefaniuk
commented
2 years ago This case study reads really well. Nice work!
Closed chrisclarknhsnet closed 2 years ago
stefaniuk
commented
2 years ago This case study reads really well. Nice work!
chrisclarknhsnet
commented
2 years ago No I don’t think it is. I’ll amend the markup to keep it simple
Regards
Chris
Chris Clark Technical Lead Product Development – Cyber Security @.**@.> 07779 289399
From: andyblundell @.> Sent: 17 February 2022 17:39 To: NHSDigital/software-engineering-quality-framework @.> Cc: CLARK, Christopher (NHS DIGITAL) @.>; Author @.> Subject: Re: [NHSDigital/software-engineering-quality-framework] Issue #244 Initial draft of performance testing page including summarized case study of DSPT use of APDEX (PR #245)
@andyblundell commented on this pull request.
In practices/performancetesting.mdhttps://github.com/NHSDigital/software-engineering-quality-framework/pull/245#discussion_r809306002:
+
+
+As such APDEX can help us answer and take action (e.g. fail the pipeline) on such fundamental questions as:
+
+* Are our users happy?
+* Have we made performance worse?
+* Would our users become unhappy when there is an increased load?
+
+## A case study
+
+For the Data Security Protection Toolkit (DSPT) we decided to use APDEX so that, prior to a fortnightly release, we could answer the question:
+
+> Has this release made the performance of the system worse?
+
+### A case study (know your audience)
+
I'm not sure this is rendering how it was supposed to...?
— Reply to this email directly, view it on GitHubhttps://github.com/NHSDigital/software-engineering-quality-framework/pull/245#pullrequestreview-886325176, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AMQ3RH3WTGK6HISKA4LYGQTU3UXCJANCNFSM5OSHN4KA. Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub. You are receiving this because you authored the thread.Message ID: @.**@.>>
This message may contain confidential information. If you are not the intended recipient please: i) inform the sender that you have received the message in error before deleting it; and ii) do not disclose, copy or distribute information in this e-mail or take any action in relation to its content (to do so is strictly prohibited and may be unlawful). Thank you for your co-operation.
NHSmail is the secure email, collaboration and directory service available for all NHS staff in England. NHSmail is approved for exchanging patient data and other sensitive information with NHSmail and other accredited email services.
For more information and to find out how you can switch visit Joining NHSmail – NHSmail Supporthttps://support.nhs.net/article-categories/joining-nhsmail/
chrisclarknhsnet
commented
2 years ago Yep no problem, agree with that just wasn’t sure where to place them and there was one image already at the root of practices…. So I’ll move that one as well.
Regards
Chris
Chris Clark Technical Lead Product Development – Cyber Security @.**@.> 07779 289399
From: Dan Stefaniuk @.> Sent: 17 February 2022 16:59 To: NHSDigital/software-engineering-quality-framework @.> Cc: CLARK, Christopher (NHS DIGITAL) @.>; Author @.> Subject: Re: [NHSDigital/software-engineering-quality-framework] Issue #244 Initial draft of performance testing page including summarized case study of DSPT use of APDEX (PR #245)
@stefaniuk requested changes on this pull request.
Would it be ok to change the location of the new images and the naming convention for the files, please? E.g. practices/images/dspt-case-study-aggregated-apdex-scores.png
This reads really well. Nice work!
In practices/performancetesting.mdhttps://github.com/NHSDigital/software-engineering-quality-framework/pull/245#discussion_r809203810:
@@ -0,0 +1,110 @@
+# Performance Testing
Can this file be renamed to performance-testing.md, please?
In practices/performancetesting.mdhttps://github.com/NHSDigital/software-engineering-quality-framework/pull/245#discussion_r809269443:
+<img src="./dsptcasestudy-scenarios.png" style=" alt="DSPT user scenarios" />
+
+
+Previously we had defined a list of user scenarios for the typical actions undertaken on the system which we named after Mr Men characters. We also defined for every 100 users how many (i.e. what percentage) would be likely to be performing a given Mr Man scenario
+
+We used these scenarios to define our thread groups within JMeter and decided we would run our performance tests for 250 users at a time which would constitute a heavy load for the system.
+
+### A case study (what does good look like?)
+
+We decided that we wanted to know if the performance for a particular scenario had degraded by more than 5% compared to previous average performance. If it did we wanted to fail the pipeline so we could investigate any new pieces of code further.
+
+### A case study (approach)
+
+Although you can apply APDEX figures to JMeter it only calculates them per endpoint whereas we wanted to aggregate our APDEX figures at the Mr Man scenario level.
+
+We therefore wrote a Python program which would take the raw JMeter results file (a sample of shown below) and using regular expressions to group results by thread names matching a Mr Man scenario, would calculate the aggregate APDEX score per scenario.
Would you be happy to include your Python code and configuration that could be helpful to others as part of the case study?
— Reply to this email directly, view it on GitHubhttps://github.com/NHSDigital/software-engineering-quality-framework/pull/245#pullrequestreview-886178569, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AMQ3RH65G7RR6QIIT2UV6Q3U3USMTANCNFSM5OSHN4KA. Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub. You are receiving this because you authored the thread.Message ID: @.**@.>>
This message originated from outside of NHSmail. Please do not click links or open attachments unless you recognise the sender and know the content is safe.
This message may contain confidential information. If you are not the intended recipient please: i) inform the sender that you have received the message in error before deleting it; and ii) do not disclose, copy or distribute information in this e-mail or take any action in relation to its content (to do so is strictly prohibited and may be unlawful). Thank you for your co-operation.
NHSmail is the secure email, collaboration and directory service available for all NHS staff in England. NHSmail is approved for exchanging patient data and other sensitive information with NHSmail and other accredited email services.
For more information and to find out how you can switch visit Joining NHSmail – NHSmail Supporthttps://support.nhs.net/article-categories/joining-nhsmail/
As per discussions with Andy