gritcsenko
commented
11 months ago Continue Exploring Engineering Excellence:
Open gritcsenko opened 11 months ago
gritcsenko
commented
11 months ago Continue Exploring Engineering Excellence:
Engineering Practices in Agile-Driven Software Development
As more and more companies strive to increase their agility in response to changing market demands, software development teams are tasked with speeding up the development process and reacting quickly to changing business requirements without sacrificing quality. To meet these goals, teams must map out a software development life cycle (SDLC) — the phases a project undergoes end-to-end—to manage resources, control costs, anticipate possible risks, and track the progress of each process required to develop a high-quality product.
There are many approaches to the SDLC, but two of the most common are waterfall and a Scrum-based Agile SDLC. Every SDLC includes certain baseline elements, but because every project is different, the SDLC will also vary to reflect specific client needs and priorities.
In this lesson, you’ll explore how Agile principles impact the SDLC and identify how a team supports effective SDLC implementation. You will discover the engineering excellence practices, processes, and tools crucial for successful product delivery.
Basic SDLC
A typical SDLC is comprised of six phases: Requirements, Design, Software Development, Testing, User Acceptance Testing (UAT)/Staging, and Maintenance. Depending on your client’s needs and project requirements, your SDLC phases might be referred to by different names or happen in a different sequence. The phases may also vary in duration and/or number.
Requirements
The objective of the first phase is to gather the customer’s initial requirements and expectations for the solution that the team will be creating.
The Requirements phase also includes defining the resources required to build the software.
Design
Once your team has a clear understanding of your customer’s technical and functional requirements, the Design phase includes more detailed work on the project, including a feasibility assessment. A team designs exactly how the system will look and function: its elements, components, security level, modules, architecture, various interfaces, and data types.
Software Development
The Software Development phase results in producing a system or a prototype that other team members can test. This phase covers the actual coding process, which takes a lot of the team’s efforts.
If the system includes hardware, its configuration and tuning for specific requirements and functions also happen in this phase.
Testing
The Testing phase ensures that the developed system or prototype meets the requirements gathered at the beginning of the project.
During this phase, a team records, tracks, corrects, and retests any defects until the product meets quality standards and all functions work as expected.
Testing activities can be manual or automated, and they may be performed in a specific simulation environment.
UAT/Staging
During the UAT, or Staging, phase, selected business users test the solution and deployment procedures in an environment that resembles the production environment as much as possible. When UAT is complete and key stakeholders have signed off on the functionality, the software is released to production.
Maintenance
After the solution is released to end users, there still could be bugs the software development team did not identify during the previous phases. If bugs occur, the team must fix the issues and repeat the Testing and UAT/Staging phases.
The Maintenance phase also encompasses planning additional features for future releases. These tasks can lead to starting a new SDLC.
When your client wants to enhance their business agility, your team can help them reach this goal by modifying the typical SDLC by applying practices, processes, and tools that support Agile principles.
Building an Agile SDLC
An Agile approach throughout a project enables your clients to react quickly to end users’ requests and provide a high-quality product that satisfies customer needs and expectations.
The Agile Manifesto highlights how engineering practices help to enhance agility in the long term.
While it is a common misconception that engineering practices are more important than the project outcome, focusing on client's goals is the way to go. Your client’s requirements should be your team’s top priority, but you must also ensure that the product meets the standards of engineering excellence to make the project successful in the long term.
Non-optimal technical solutions might be deemed acceptable as a temporary way to meet important, short-term project goals, but these solutions are not ideal and won’t work in the long term.
Making your SDLC Agile will empower the team to incorporate regular client feedback flexibly, prioritize the most valuable features, and produce a testable outcome. To reach these goals, your team should take two actions.
Make changes faster without increasing costs or degrading quality.
Your client’s primary goal is to provide a high-quality product that will increase user satisfaction, bring value as soon as possible, and help the company gain a competitive market advantage. Clients also want to keep the software development process cost-effective.
Your team can help your clients reach those goals by prioritizing the introduction of changes to address any current solution gaps and improve product quality effectively. Regularly making changes saves time and money in the long run, while postponing changes can have a critical impact on your project—like having to rebuild your solution at a later stage.
Produce a potentially shippable (ready for detailed customer feedback) product increment at the end of each iteration.
In Agile-driven software development context, a product increment stands for the outcome of each iteration (e.g., a new feature) that is ready for testing.
Regular feedback from your client is crucial to ensure that you are on the same page about solution requirements. Your client might add or modify requirements at some point, and your team must be able to adjust quickly. Your team can prepare for quick adjustments by delivering a product increment with each iteration (e.g., a new feature) that is ready for client testing and a potential release.
This action is useful not only for trying out a real product with end users but also if your team needs to create a minimum viable product (MVP) to test a business idea. Engineers can use the feedback they receive on this bare minimum version for further improvement of the product.
These two actions are the foundation of Agile-driven software development. Taking these actions requires your team to follow certain software development principles.
Key Principles of Agile-Driven Software Development
Many software development principles are vital to upholding the engineering excellence standards applicable to any SDLC. Some of them are especially important when your team aims to make changes faster without cost and quality degradation and produce a potentially shippable increment at the end of each iteration. These four principles are the foundation of an Agile SDLC.
Prioritize Code Quality First
Following best code design principles ensures the team will have fewer issues to address at a later stage. As a result, a client can enjoy a high-quality solution faster and cheaper.
Shift Feedback Left
Shift all feedback you receive—from your customer, team members, and automated verification processes (e.g., unit tests, static code analysis)—to the left: the earliest point possible in the SDLC.
Seeking feedback earlier will make it cheaper for your team to fix any issues and resulting negative outcomes and enable your team to produce higher-quality code.
Automate All Repeatable Tasks
Automation saves time that developers can use for other tasks, which increases their productivity. Engineers can automate many repeatable tasks in the SDLC to receive reliable and fast feedback on the existing functionality. For example, your team can automate code quality checks, test execution, report generation, metrics collection, and more.
Follow Lean and PDCA Practices
Your team should continuously strive to assess your current software development processes and identify ways to improve them further. Lean and PDCA are two practices that directly support development speed and agility.
The term “lean” was coined by a research team headed by Jim Womack in the late 1980s to describe Toyota’s business practices. Lean stands for a set of practices and principles that can help your team maximize the value you bring to your customer while minimizing waste (non-optimal processes that add no value to the customer’s product or service). Examples of waste are useless meetings, tasks, and documentation, and ineffective working methods, such as multitasking. Lean encourages Agile engineers to work in iterations and get feedback on their work faster, enabling them to test business ideas with the help of MVPs. Lean applies to all SDLC phases.
The “Plan-Do-Check-Act” (PDCA) approach starts with planning a new practice that your team considers necessary at a certain point or checking a hypothesis you have. Then you implement your plan, study how it works in a real environment, and analyze whether the implemented feature brings the expected results. Finally, you act based on the lessons learned in the previous step. If something went wrong, your team should start another PDCA cycle with a better plan. But if the initial plan was successful, you can incorporate your findings into the system and plan new improvements. To make PDCA effective, everyone on your team should contribute to it. Retrospective meetings are the right time to conduct this practice regularly.
Your team should integrate these software development principles into your project practices by adopting a CI/CD workflow.
Implementing the Key Principles with CI/CD
Continuous integration (CI) goes hand in hand with continuous delivery (CD). Together, they represent an approach software development teams use to support an Agile SDLC.
CI/CD embodies a practical implementation of key Agile SDLC principles. For example:
In short, employing CI/CD practices helps your project become more Agile.
Taking an Agile Approach to a Software Development Project
When it comes to being responsive to business changes, an Agile approach to a software development project is much more flexible and effective than the traditional waterfall approach. The phases in a waterfall SDLC happen in a linear sequence. In contrast, an Agile SDLC might have several implementation versions. For example, the Scrum-based Agile approach has short iterations and allows several phases to happen simultaneously.
While it is impossible to describe a universal SDLC that reflects the specifics of every project, it is still important to imagine how a team might go through SDLC phases in terms of a timeline and iterations in every phase of a project end-to-end. The example below is based on the CI/CD approach within an Agile-driven environment. It illustrates when and how a team can implement Agile engineering practices, processes, and tools to ensure successful delivery.
This sample project is conducted using the Scrum Framework on a two-week iteration basis. It supports the Agile principles of receiving and being able to implement valuable feedback faster. The stages the team follows are not linear—for example, when team members finish the Requirements, Design, and Planning stages and begin the Development and Testing stages for the current iteration, they also start working on the Requirements, Design, and Planning for the next iteration. Engaging in software development activities in parallel helps make your work as efficient as possible.
Applying Engineering Excellence practices, processes, and tools during the Development and Testing phases in this baseline project is essential for successful delivery.
The Development Phase
An Agile software development team usually conducts development and testing activities simultaneously. Pieces of code can be tested as they are completed, which provides the early feedback required to improve other development outcomes. The Development phase covers pre-coding activities, coding, and code review. Throughout this phase, a software development team is also involved in other development activities: knowledge sharing and management of technical debt.
Pre-Coding Activities
Before your team starts coding, you must check all existing tasks against the Definition of Ready (DoR) criteria to ensure that the requirements are clear and your team will be able to measure each task’s completion.
This preparatory work helps ensure that the team is on the right track and will be developing the requested functionality instead of wasting time and effort to produce an increment with unexpected behavior. As a result of this step, a team is ready to start the implementation of tasks.
Coding
Usually, software developers divide larger coding tasks into smaller ones that they can complete in one or two days. To ensure production-ready code, developers should:
After developers take these actions, the code is ready for review.
Code Review
Code review is mandatory to perform before integrating a code change to a common source code. By reviewing code for possible defects and coding standards violations that the automated verification does not cover, developers ensure high code quality.
Other Development Activities
Throughout the Development phase, team members should actively participate in knowledge sharing activities and managing their technical debt.
To consider a coding task done, a team follows the definition of done (DoD) for development activities, which stands for a set of rules, checks, and quality gates that a task should pass. After a task passes the DoD, the development phase is over, and testers can start verifying the piece of code.
The Testing Phase
The Testing phase covers pre-testing activities, automated and manual test execution, and defect management.
Pre-Testing Activities
Pre-testing activities start at the same time developers begin the implementation of their tasks. Using special test case management tools, software testing engineers create test cases for new functionality covering all possible business scenarios. Their goal is to help identify as many defects as possible and speed up feedback when testing.
Once the test cases are ready, the testing engineers start to implement “to-be-automated” test cases. Software developers can support them in these activities.
Automated and Manual Test Execution
Once code is ready for testing, testers deploy it to a Quality Assurance (QA) environment. Immediately after that, they execute automated tests to ensure the functionality is not broken and is suitable for further manual QA checks.
If the results of the automated tests are positive, manual QA engineers perform two types of functional testing: using previously written test cases and exploratory testing. (Testers perform exploratory testing with the help of test scenarios that they create instantly—when they are actually checking the system.)
After that, QA engineers conduct non-functional testing that verifies specific aspects of the system, including performance, security, and usability.
Defect Management
QA engineers can identify defects by analyzing the results of both automated and manual testing. They submit the defects into the defect management system so that the developers can fix them, ideally during the same iteration. It is also important that QA engineers cover those defects with automated tests to prevent their re-appearance.
Once developers fix the defects, QA engineers verify the results.
When product increments successfully pass QA engineers’ verification, the code is ready for the next phase—UAT/Staging—where business users check it in a staging environment and/or in production.
Agile projects combine business and development teams that work together to enable a successful product delivery.
Example of a Project Ecosystem
A project ecosystem is a technological environment that provides necessary tools and infrastructure that help a team perform their tasks efficiently. The example project ecosystem below represents how a team can properly implement CI/CD processes to enhance the agility of their software development process.
A typical cross-functional team working on an Agile-driven software development project would include a product owner on the customer side, and a project manager, developers, manual and automated testers, and system engineers on the development side.
Artifact Repository
An artifact repository stores builds (compiled source code). The latest artifacts go from this repository to the QA environment.
Testers
Testers write test cases and run them in the QA environment to ensure final pieces of code are defect-free.
System Engineers
System engineers implement core engineering activities throughout the entire SDLC. Their tasks include:
System engineers also perform infrastructure (related to repositories and servers/environment) management tasks by:
Quality Assurance Environment
The QA environment simulates the production environment. At this stage, testers perform automated and manual checks. Based on the results of these checks, your team decides if a solution is ready to go to UAT or needs to be fixed first.
There may be one QA environment or multiple environments (if separate environments are required for different types of testing).
CI/Build Server
The build server, also called the continuous integration (CI) server, is a centralized environment for building project code. If the build is successful, the system automatically runs code analysis and unit tests and returns the feedback to the software developers. If the server identifies no severe issues, then the build is transferred to the artifact repository. Otherwise, the server sends the feedback to a developer, who fixes the issues.
Business Users
Business users, or customer representatives, perform user acceptance tests to ensure that the solution fully complies with their requirements.
Software Developers
Software developers perform core engineering activities, mostly during the Development phase of the SDLC. They write unit tests and use static code analysis to ensure high-quality code.
Developers incorporate the feedback from all automated and manual testing activities. They may also support testers in writing automated tests.
Version Control
In this code repository, developers store their code to enable tracking and managing changes.
Code Review
Code review is conducted by developers, along with their peers and managers, usually with the help of a version control system or dedicated system for code review.
User Acceptance Testing/Stage Server
The User Acceptance Testing (UAT)/Stage environment closely simulates the production environment. On the UAT server, business users or the client’s representatives perform user acceptance testing.
Stakeholders review the results of user acceptance testing to decide whether a product can be released to production. Although there may be minor bugs present, a solution shouldn’t have any severe issues.
End Users
End users are people who use the final product.
Product Owner
A product owner manages the product backlog and prioritizes its items for the software development team. This person also shares client requirements and feedback with the team.
Project Manager
Ideally, an Agile team has no project manager. However, complex projects require a dedicated role of a project manager who can perform as a SCRUM master on a development team. The project manager is responsible for the successful delivery of the project and ensuring the team follows all SDLC processes properly and timely. This person closely communicates with the client and across the multifunctional teams to ensure everyone is on the same page.
Production Server
A production server is a live environment for the solution used by the end users.
Close cooperation and transparent communication among all team members are essential to reach agility goals. Agile principles can help your team create an SDLC that fully matches your client’s goals and gives them a competitive business advantage.
Evolving Engineering Excellence
Now that you’ve discovered more about various engineering practices in Agile-driven software development, you are ready to deep dive into the different SDLC phases in the following lessons.
Remember that staying agile will help you effectively organize your tasks throughout the project and deliver value to your client irrespective of changing business environments.
Best Practices for Ensuring Agility of Your Project
To ensure agility on your projects, your team must follow these best practices: