Retrospectives is an Azure DevOps extension to perform smart and efficient retrospectives from within the Azure DevOps pipeline.
Retrospectives are an important practice in becoming an effective team, allowing the team to gather feedback and continuously improve based on the feedback.
Research from the 2018 State of DevOps report indicates that Elite teams are 1.5 times more likely to consistently hold retrospectives and use them to improve their work. Furthermore, a 2013 meta-analysis on teams indicates that teams that effectively debrief/conduct retrospectives are 20-25% more effective.
Teams often use external retrospective tools, white boards with Post-its, OneNote, etc. to conduct retrospectives. The data then needs to be collated, and any actionable items need to be created in your Azure DevOps pipeline.
The Retrospectives extension allows you to do all this and more from within Azure DevOps.
The extension can be installed from Azure DevOps Marketplace.
Please note: the screenshot examples use light mode and dark mode interchangeably because both are supported!
Navigate to the Azure 'Boards' tab in your account on the left hand navigation. Select the 'Retrospectives' tab under 'Boards'.
You are now on the Retrospectives page. Select your Azure DevOps team from the selector in the header.
Create a new retrospective using the 'Create Board' button. If this is your first retrospective for your selected team, then press the "Create Board" button in the center of the screen.
If you have created a retrospective before for your selected team, then you can select "Create a new retrospective" from the navigation drop down.
When you select New Board or Create new retrospective as above, you will see the following dialog:
Please enter the appropriate information:
Note: Retrospective Title is the minimum 'required' information (other fields can stay at default as needed). Once the title is provided, the Save
button is enabled. Save the retrospective using the Save
button.
Once you have created the retrospective board and you want to select a retrospective board different from the currently displayed board, click on the retrospective board name and select the desired one. You can use the search box to find the appropriate retrospective boards if you have a large number of boards.
Once you create a new retrospective, you can share a link to it to all participants. Users can then access that link even from their mobile browsers to participate in the retrospective. The extension offers real time synchronization, so all users will see the most up-to-date information without having to refresh the page.
The number of columns in each retrospective is determined by the selection made when making the new board. Regardless of the template, they still follow the same sequence of phases.
If you had selected the 'Display 'Retrospective Prime Directive', you will see the option to view that directive which sets the stage for the retrospective. It is recommended that you click on the Prime Directive and read it out loud for everyone to hear. Remind everyone that any issues discovered will be assumed to be process problems, not people problems.
In this phase feedback is collected from all participants. Users can add feedback under any of the columns using the 'Add new card' button or by double clicking the empty space of the column's background.
If when creating a retrospective, the checkbox for "Do not display names on feedback" is checked, then the cards will show no names, appearing anonymous.
If when creating a retrospective, the checkbox for 'Obscure the feedback of others during the Collect Phase' was checked, then while typing, the feedback of others will not be shown, even if anonymous. You can only edit items that you have created in this mode.
Once feedback from all users is collected, move on to the next phase of Group.
In this phase, any similar feedback can be grouped together beneath a "header" or champion item. If you feel 2 feedback items are similar, drag one onto another to group them together. Dragging any item onto a group, will add items to that group.
If you would rather not drag you can use the feedback item's action menu (the ellipsis '...') to "Group Feedback". This will bring up a search box and you can type in the feedback item you would like to group your selected item under.
Both methods of grouping will result in the item being grouped beneath the other.
Once all similar items are grouped together, move on to the next phase of Vote.
In this phase, participants will individually go through all the feedback items and vote on the ones they feel are important, by clicking on the 'Upvote' icon. Users can reduce their number of votes on a specific item by clicking on the 'Downvote' icon.
Note the Max Votes per User set by the board creator are displayed as the denominator in a ratio of "Votes Used". Votes used by the user are updated in real time as the user clicks on 'Upvote' and 'Downvote' icons.
Once everyone is done voting, move on to the next phase of Act.
In this phase, the team will go through each feedback item. You will notice that the items with the most votes will appear at the top of the column. It is recommended that each time you do this exercise, you only select one or two of the feedback items to take action against. Click on the 'Add action item' button on a feedback card, and select the type of work item that needs to be created in Azure DevOps.
This will open up the standard Azure DevOps work item creation form. Enter the work item details and save. This will create the work item in your Azure DevOps account and also associate it to the feedback item.
You can optionally include a "Team Assessment" that anonymously collects feedback from participants. To get the feedback form, you can click on the "Team Assessment" link at the top of the board.
The assessment tracks 6 categories:
The users will rank on a scale of 1 to 10; scores of 1-6 are categorized as "Unfavorable", 7 and 8 are "Neutral," and 9 and 10 are "Favorable."
Once the team assessment has been completed, the summary of answers can be viewed in the Retrospective Summary Dialog (see below).
Teams may choose to address the issues found in the team assessment in multiple ways:
Create a new retrospective using the steps above with the following exceptions:
In this phase the facilitator sets the stage for the retrospective. It is recommended that you click on the Prime Directive and read it out loud for everyone to hear. Remind everyone that any issues that discovered will be assumed to be process problems, not people problems.
In this phase each team member clicks on the Team Assessment and anonymously answers all questions and hits "submit". It is critical that you remind everyone that all feedback is anonymous and confidential.
Throughout this phase you will periodically check the retrospective summary.
If the majority of the team hasn't responded to the assessment, close the summary dialog and encourage everyone to participate, reminding them that if their voice isn't expressed in the assessment it can't be acted upon.
Re-open the retrospective summary and when the majority of the team has responded, discuss which dimension the team would like to improve upon. Note: If more than 20% of the responses are unfavorable for a dimension, we recommend focusing on that area first. Otherwise, it is beneficial to focus on psychological safety first. This is because without psychological safety the more difficult conversations that need to happen to make improvements in the other dimensions will not happen.
Once the team has decided which dimension to focus on, the facilitator will click the button "Discuss and Act" which aligns to the dimension the team wants to focus on. This will create retrospective with a template that aligns to the dimension.
In this phase team members will provide feedback to help improve the dimension they've chosen to focus on. Ask team members to input feedback cards on the left 2 columns.
After a few moments, the facilitator reads the cards outloud. When it's apparent that no additional feedback cards are being added, the facilitator asks the following question: "Reflecting on the feedback we've collected, if you could take only one action in the next sprint that would yield the biggest improvement on [dimension team is focused on improving from Team Assessment] what would it be? Capture this in a feedback card in the third column (One action in the next sprint)"
In this phase the team will group similar items together. To expedite the process we recommend only grouping the action items together in the last column. After the majority of the team has added feedback cards to the third column, the facilitator switches to the "group" tab and collectively groups like ideas together with the team (see above).
In this phase the facilitator asks participants to vote only on the third column and the top items for improvement are identified. The facilitator then waits for votes to compile and after the majority of the team has voted (the facilitator can view the retrospective summary to quickly determine how many votes have been cast), the facilitator clicks on the "Act" tab which automatically sorts the cards from the most to the least voted upon.
Follow the steps above to assign actions as work items in Azure DevOps. Note: We recommend only selecting 1-2 actions per sprint to ensure completion prior to the next sprint.
The Retrospective Summary dialog is accessible from the "Board Actions" menu in the top navigation.
Once there, there is a summary of the Retrospective. This includes:
If there is a team assessment, then a charted summary will show the vote breakdown by favorability, along with an average score.
There may be times that a team may want to reflect on past retrospectives they have run. There is a History tab in the top navigation of the tool, next to "Board" header.
From here, one can look at past retrospectives and see a few high level stats and the action items created after each by clicking the individual retrospective.
Planning in advance will help you make the best use of the team's time in completing a Team Assessment Retrospective.
When | Action |
---|---|
Week in advance (more or less depending on your team’s rhythm of business) | Create and send a calendar invite for 50 minutes for the Team Assessment. Make sure that all team members can be present. |
3-5 days prior | (1) Create the Retrospective in ADO; (2) Create the schedule and timebox each section; (3) Update the meeting invite with the schedule; (4) Review action items from previous retro to familiarize yourself and be prepared to share |
Activity | Time | Guidance |
---|---|---|
Welcome & Kickoff | 10 min | (1) Welcome the team; (2) Review Agenda; (3) Review actions from last retro and make any updates or clarify next steps; (4) Kick-off with the retro |
Conduct Team Assessment | 5 min | Share link and give team members 2 minutes to complete team assessment |
Identify area for deep dive | 10 min | Discuss results and agree on area for deep dive |
Conduct the deep dive retro | 15 min | Collect, Review & Group Columns 1 & 2 |
Determine action plan(s) | 10 min | Collect, Review, Group and Vote on Column 3 |
Close the meeting | 2 min | Thank attendees for participation and summarize |
Giving kudos, or recognizing others for their achievements, is one of the highest drivers of employee satisfaction and retention. When people feel appreciated and supported, they will be more engaged and motivated. Who doesn’t want that? You have an opportunity to provide a space for kudos in every team retrospective. Make sure to add a column for “Kudos” in your retrospective board. It is that simple and it just takes a moment for people to add a few pieces of feedback to acknowledge the great work of others. When you finish your retrospective, make sure to read through each kudos feedback item and take a moment to celebrate!
Team Working Agreements are shared commitments to ways of working together. They establish team members' work preferences. A shared agreement will balance individual and team needs, and set clear expectations for how the team will work together.
A good team working agreement must include everyone's background, needs and voices so a retrospective is a great way for a team to collect that information and then bring it together. The agreement should be relatively short, 5-7 items.
Bring your team together and create a retrospective board with 1 column:
Follow the phases through from Collect, Group and Vote. Make sure every team member contributes feedback items to the Board before you Group and Vote. Read every card outloud before Grouping like items together. After voting, you will see those working preferences that got the most votes come to the top.
Select the top 5-7 items and talk through them together as a team to ensure that they are clear and that everyone agrees. Put those items in a document that you can keep in front of your team (at the top of a Microsoft Team Channel, or in your Azure DevOps Dashboard). Review the Working Agreement at the beginning of Team ceremonies or events. Whenever there is a change in the team composition, the Working Agreement should be reviewed again to ensure that all team members agree to each item.
Below are a number of best practices for driving team effectiveness and using the “Team Assessment” measurement within the Retrospective Board.
Copyright (c) Microsoft Corporation. All rights reserved.