DevOps Mutualization - Structuring conversations around SDLC and Iterating the different types of evidence that needs to be produced

[mcleo-d]

Description

This issue has been created to capture and iterate the compliance evidence required by banking and fintech DevOps teams.

DevOps Mutualization Meeting Notes

Date and Time : Thursday 30th July @ 1pm ET / 6pm BST - https://github.com/finos/community/issues/52#issuecomment-669343645

• Software Supply Chain with Grafeas and Kritis introduced was shown via a slide.
• The question was asked, "How do we create a taxonomy around the metadata to gather a software supply chain?"
• It was explained that Grafias stores information and validity tests whilst Kritish enforces policy for the data stores in the Grafias database.
• It was asked whether a straw man of common pieces of metadata should be created that could include the aspects below ...
  • Record of code review/4-eyes check
  • Record of test execution
  • Record of test result acceptance/sign off
  • Record of code/image/vulnerability scanning
  • Environment deployment history/promotion
  • Record of ITIL related control points
  • Conformance to other control points – CSO or Architecture compliance
  • Code/config changes - commit id/PR/etc
  • Traceability to requirements/jira issues, etc
  • Test plan
  • Change classification - material/minor - impact and testing scope, risk, etc
  • Change/risk assessment - maybe some automated risk assessment, blast radius assessment"
• The group was then asked, "Do these common factors sound reasonable and what else can we add to the list?"
• It was agreed all points strike as necessary and certain teams like to gather evidence surrounding the changes that get added to code.
• It was also pointed out that change types and requirements are missing from the list.
• A question of the "artefact types" expected to be collected was asked and also items like "technical design documents"?
• The group fed back collecting information makes a lot of sense.
• It was agreed that we need to collect the information, change types and audit trail related to all software changes.
• The group was asked, "Is all the information actually needed as not every commit is a perfect business requirement and edge cases are difficult to track?"
• It was explained the change type does often influence how software changes move through the release pipeline.
• It was explained there is work being done on how to identify material change as audit trails should change to something more reliable than human, risk based, self categorisation.
• The approach should be more artefact related in order to reduce the risk surrounding making a change.
• The question "Is there some form of automated risk assessment that improves the speed of deployment whilst reducing risk to live systems?" was asked to the group.
• It was added that people want to know the blast radius of change in any given environment and architecture often influences the blast radius. Tightly coupled architectures have a greater blast radius to microservices.
• It was also added, items not thought through whilst coding are the areas that hit teams. Hitting deadlines with last minute changes are also the types of behaviour that increases risk of breakage.
• The group fed back that ticking boxes manually on change requests is a horrible way to determine success as too much paperwork with standard responses. The intent behind the change process is good, but the mechanism is wrong.
• The group was asked, "Do we also need to consider what is a release and what's the difference compared to a commit?"
• It was noted the team should capture the "What" around the topics and and focus on the "How" at a later date whilst implementing the change.
• Group members agree the topic is close to their heart and an initial set of taxonomy is useful as we won't need to reinvent.
• The question "Is there a good way to store and share the information?" was asked to the group.
• A clarification of sharing artefacts or sharing policies was asked as risk and policy teams have implemented homegrown systems across 20 years to collect evidence.
• These teams are now moving towards industry solutions like JIRA and Service Now where attachments can be added to immutable stores.
• The question, "What are the things, how are we tackling, should we share notes and get regulators involved in a more automated way?" was asked to inform a standardised way of presenting the information across banks so regulators can understand.
• It was added that we should go above what the regulator wants and build a secure pipeline that reduces lead times from a security point of view as security requirements are more strict as they want demonstrable proof.
• The group agreed we should show the regulators what good looks like.
• It was explained that many silos have been removed through DevOps but there are now lots of data silos. Getting visibility across data is important to consider.

[mcleo-d]

Hey all - The following Evidence Lake Document has been created in the DevOps Mutualization Project on GitHub to break the conversation out of this issue and place it in project where people can add their own documents and edit existing ones through pull requests.

https://github.com/finos-labs/devops-mutualization/blob/master/docs/evidence-lake.md

Let me know if you have further questions.

James.