irisoren-sg
commented
2 months ago - [x] #140
- [x] Edit introduction: 7.1 currently reads "It sets out recommended practices for documenting the Design as an Analytical Plan, making changes to the plan and keeping an audit trail of those changes, testing for reproducibility and uncertainty, and agreeing this with the Approver. An Analysis Steering Group can be an effective means for communications about the Design." The rest of the chapter is sparse on details about "keeping an audit trail of those changes, testing for reproducibility ". Can we remove the entire sentence from the intro, or edit to match the most significant points covered in the chapter?
- [x] Edit "7.3 Testing the methodology":
This section covers a lot of points, and needs to be reworked.
- "If the analysis is written in code, this should include appropriate testing to ensure that the code works as intended to deliver expected outcomes. Following recommended good practices for code development is the best way to ensure that code is fit for purpose." This confounds testing of the design with assurance of the analysis.
- Do we want to change this to "Assuring the design"
- Much of this paragraph does not speak to the subheading
- "If the analysis is written in code, this should include appropriate testing to ensure that the code works as intended to deliver expected outcomes. Following recommended good practices for code development is the best way to ensure that code is fit for purpose." This confounds testing of the design with assurance of the analysis.
- [x] Documenting the design:
- Current: "Best practice is to document the design process recording how the proposed analytical process generates the requested insights. This plan should be supported by the design documentation." Change to "The design process should be documented in a proportionate manner. A design document that records the Analytical Plan should be produced by the Analyst, and signed-off by the commissioner"
- If 7.4 is included then "Documentation of the Analytical Plan should be proportionate to the Design. For larger or more complex projects, the Analyst might request assurance of the Analytical Plan." can be removed from 7.2
- Documenting the design should be added to Analyst responsibilities
- [x] Check that there is reference to ethics in design and add if needed
- [ ] Add information about "Assurance plan". The Analysis section reads "The analyst will also carry out the assurance plan and shall provide traceable documentation of the assurance they have undertake" This is the first mention of the "assurance plan" outwith the definitions. So we should add details of Assurance Plan to design section. Add hyperlinks from Analysis assurance plan to either first reference, or definition
- [x] Approver (and assurer) responsibilities repeated from Chapter 6 . Chapter 6: "The assurer should understand the assurance requirements and plan time in for the assurance activity. The Approver should document the new project and ensure that required mechanisms are in place for the appropriate assurance to take place. For example, they should ensure that the analyst and assurer are aware of local assurance protocols." Chapter 7: "The Assurer should agree with Analyst and Commissioner what the assurance requirements will be and plan sufficient time for the assurance activity. The Approver should document the new project and ensure that required mechanisms are in place for the appropriate assurance to take place. For example, they should ensure that the Analyst and Assurer are aware of local assurance protocols"
- [ ] Production of analytical plan and assurance plan as outputs. These are referred to in Analysis and key concepts
- [x] Design considerations for multi-use models
- [x] Remove paragraph "The Commissioner remains important through the analysis cycle..." as not relevant to the design phase
- [x] Simplify materiality paragraph and move to "Designing the analysis"
- [x] Move paragraphs "On completion of the design phase.." and subsequent to "Designing the analysis"