Team-apps JIRA workflow

[Mohammer5]

There's no documentation of how our workflow works in-detail, so I'd like to sum up what I've understood so far to see whether I'm right or not. Eventually the result of this discussion should turn into docs somewhere prominent and easily accessible for team-app devs!

1. Todo

• Everything in here is prioritized and ready to be picked up.
• Normally it should not have an asignee
• It can have an assignee when
  • The issue came back from testing
  • It's clear who'll do that issue anyways

There's nothing to be done here by devs other than picking an issue when looking for work. When picking an issue, the dev should

1. assign it to him-/herself
2. move it to "In progress"

2. In progress

• Issues in this column should
  • be actively worked on
  • or on hold because it's blocked by sth else after having started to work on it (This means that blocked issues that haven't been worked on yet should be in "Todo")

Once a developer is done working on an issue and has created a PR, before backporting, adding JIRA test issues and updating the docs, the issue is moved to "In review" and assigned to another team-apps developer who should review the issue

3. In review

• Issues in this column should at least have a PR attached to them
• The issue will bounce back and forth between the developer and reviewer whenever something is missing until all requirements are fulfilled (JIRA test issue, updated docs, backporting), always including a comment by the reviewer what is missing

For some reasons issues should move to "In review" before adding tests and docs:

1. Docs would be "live" before the issue has been merged
2. Implementation might change based on the result of the review which would require a change of the previously merged doc changes
3. Implementation might change based on the result of the review which would require a change of the previously added JIRA test issue

The reviewer should always leave a comment on the issue when assigning it back to the developer to make it clear in the issue's history what has happened. This could be "Code works, please update docs, add test cases and backport to specified target versions". Finally, after everything has been added & has been checked by the reviewer, the reviewer moved to issue to "testing", as assigning it back to the developer would be an unnecessary step

[varl]

The stale JIRA workflow is here: https://github.com/dhis2/dhis2.github.io/blob/master/_guides/jira-workflow.md

Would be good to consolidate them!

[varl]

I'll explain the filters backing our team's process as they inform the process of how issues are treated. JIRA does not have a 1:1 mapping of the states an issue has in the DHIS2 Agile process. Some of this is general to the DHIS2 Agile process and some of it is internal to Team Apps.

TL;DR

• Open = Needs triage (unverified)
• To Do = Accepted issue (verified)
• To Do + Internal rank: Prioritised issue
• Planned: Planned work
• TApps Kanban Board = Worklog
• Target version is used to indicate which versions of DHIS2 an issue should be fixed for, but isn't used for sorting issues

The full monty

As outlined above, the happy-path progression of an issue is:

**1. Open

2. To do
3. Planned**
4. In progress
5. In review
6. Testing
7. Done

To better map to the DHIS2 Agile process, let's unpack what each status in bold means:

• Open:

  • for a bug means that it is unverified
  • for a feature means that a product manager has not decided if we are going to do it

• To do:

  • for a bug means that it has been verified and should be fixed (but not when)
  • for a feature means that we are going to do it (but not when)

• Planned:

  • for all issues means that an issue is planned to be done and work is ready to begin

At this point, we have issues that we need to triage, and issues that have been triaged. In itself, this says nothing about the importance of the issues, when they should be fixed, in what order they should be fixed, nor by which team that should do it. Let's address the last thing first, who is responsible for doing the work?

We use Components to indicate which parts of the system is affected by an issue. One issue can affect multiple parts of the system, e.g. both frontend and backend for an application.

🚨 Issues that contain references to multiple components should never make the "In progress" stage in their current form. Instead they should be split up into N issues (where N = Component Count) and linked together.

For Team Apps, all components that we are responsible for, I (@varl) is set to the Component Lead, to make it easier to maintain a dynamic list of our issues in a filter:

Team Apps Components

• Shows all issues that has a component where the component lead is set to varl (my username on JIRA)
• Useful for keeping a list of which components Team Apps is responsible for up-to-date automatically
• Answers the question of who is responsible for fixing the issue

The components filter can then inform our base filter, so that we have one filter which we can use to narrow our search to all the TApps issues in JIRA:

TApps Issues

• Shows all issues Team Apps are responsible for regardless of status
• Allows for combining components with other fields to indicate TApps ownership without changing the base "Team Apps Components" filter
• Useful for finding closed issues
• The base filter for most other "TApps" filters

Armed with those two filters, we can set about trying to tame our process and backlog.

Managing the Backlog

Step 1: TApps Backlog: Triage

Players: QA, PDM, TL, Developer

Responsibility

• As our JIRA is public anyone can open an issue, which means that issues that are in the Open status are issues that need to be verified before a developer starts to work on it. This is a different person depending on if it's a bug or a feature, and what part of the system it targets, but usually either a PDM (Product Manager), Developer or QA Tech can triage issues. Ultimately the PDM is responsible for the decision, so if unsure, reach out to the PDM to figure out if something should be rejected or accepted.

• The transition from Open to To Do indicates that an issue is accepted

• To reject an issue, Close the issue with the Resolution set to an appropriate one, and write a comment explaining why the issue was rejected

At this point, the issue should be in To Do, but Internal rank should be EMPTY, which indicates that an issue has been triaged and is now accepted by TApps, but not yet planned.

Step 2: TApps Backlog: Accepted

Players: PDM

Responsibility

• Lists all issues that has been successfully triaged (verified & accepted)
• Issues in this list should indeed be fixed eventually, the question of in what order and when to do so is unclear

At this point we are ready to start to expand the map we started on above, as we are now at the point that we have issues To Do, but we don't know which of them is more important to start on.

Let's add the Internal rank field to the mix.

A PDM goes through the "TApps: Accepted" filter and sets the Internal rank field to a integer value between 0-1000 which denotes the expected priority of an issue from the PDM side.

It should be no surprise that we work on the highest priority issues first, and this is a way to align the priority in a way that cross-cuts the product streams (Android, Tracker, Analytics) with our own (Apps).

Once an issue gets an internal rank, then it will disappear from the TApps: Accepted filter and show itself on the "TApps Backlog: Prioritised" filter:

Step 3: TApps Backlog: Prioritised

Players: TL

Responsibility

• Issues that are in To Do and has an Internal rank are known to us as "prioritised issues"
• We know what our priorities should be, and we know that they are accepted, so they should be ready to be worked on
• We still don't have an answer to "when" to work on an issue; since other issues might block it, etc.
• The TL (Team Lead) uses the "TApps Backlog: Prioritised" to figure out dependencies and the realistic order in which to do the prioritised issues, and sets the Sprint field to mark the issue as planned

Given three issues with an internal rank: A (1000), B (900), and C (800)

The expectation would be that we start to work on A first, then B, then C. If this was always true, we would not need Step 4, and could work straight from the prioritised backlog. Sometimes we do end up with in a situation where the PDM only cares about A so it is the highest priority, but it is blocked by B, so we have to work on a lower priority issue to get to A.

One way would be to just switch the priorities, but this would obscure the fact that the PDM really cares about A. So we plan to do B before A.

When an issue is planned it is released from the prioritised backlog, and will show up on the planned filter:

Step 4: TApps Backlog: Planned

Players: Developer

Responsibility

• Issues that are in the status: planned
• The authoritative plan for who (TApps), what (issue), when (planned), why (priority)
• This is the planned backlog for TApps and only shows issues that have not been started on yet
• The general rule is that if an issue has been started, it cannot be re-prioritised on-the-fly so if work has begun on a 700 issue, and a 900 issue enters our backlog, then work is first finished on the 700 issue before the developer picks up the 900 issue
• The next free developer picks an issue from the top of the list (highest internal rank)

Step 5: TApps Kanban

Players: Developer

Responsibility

• The filter that backs our Kanban Board
• Contains a super-set of the "TApps Backlog: Planned" filter, to e.g. show in progress, reopened, and recently closed issues for a better overview of the active worklog

[varl]

@dhis2/team-apps There's (at least) one important general rule missing in the description above:

One developer should only have one JIRA issue in progress at a time.

A core tenet of Kanban is that once work commences on a task, focus is on moving that task forward until it is done. The person who started on the issue is the "issue owner" and is responsible for progress on that issue until it is done.

This does not mean that the issue owner must do everything herself; it does mean that the issue owner should facilitate that it is possible for other developers to help out. That could mean for example to create sub-tasks or asking for specific pull requests.

However, It is easy to forget to ask for help (there's always other work to be done), so if you find yourself idle, reach out to an issue owner and offer to help out with another issue that is in progress.

At the end of the day, the Team is always responsible for all the work that we do. We stand behind our work as a unit and take responsibility for the work even if we were not the ones that implemented it.

Idleness

There are times where a developer must wait for something, e.g. for a code review, for testing, etc. It is tempting to start on a new ticket in JIRA to avoid idle time. Not being idle is good, but starting on a new issue in JIRA can be a context switch with a high cost if it means working on a different application or domain. When the first issue invariably comes back, another high impact context switch occurs. This drains mental energy and can result in negative stress as it is not a sustainable model for development work.

A sustainable pace is very important for Kanban (and me, and should be for every developer), and is fundamentally why it is not oriented around sprints.

So, if a developer is not to sit idle and wait, and not start on a new issue, what is she supposed to do? There is no clear answer, as a context switch is impossible to avoid, but we can reduce the cost of the context switch from high to low or medium by thinking about what we switch to. For example, these might have a lower cost than a new JIRA issue:

• Stay in the domain and refactor code you noticed when working on the issue
• Improve test coverage and testability for the feature, or the surrounding context
• Work on a task in a standalone library that doesn't carry the same burden of process as a JIRA bound issue
• Write documentation for the surrounding context or the feature
• Ask in the team chat if there is a small part on another issue (see above about "issue owners") that you can help out with

Or they might be higher, that is subjective. The idea is to think about it and let the cost of the context switch factor into the equation of what to work on while waiting for another issue to become ready again.

[Birkbjo]

There are times where a developer must wait for something, e.g. for a code review, for testing, etc. It is tempting to start on a new ticket in JIRA to avoid idle time.

Are we considering these as 'in progress'? In my mind we pass 'the torch' of the issue with these statuses, especially testing. So I tend to start looking into a new issue once an issue I'm working on is set to testing. I understand that I am the issue owner, and therefore overall responsible for the issue, and will follow it up if it doesn't pass testing (or review). Sometimes it takes a considerable amount of time before the tests are executed, and waiting for these to pass until I can start a new issue seems unecessary, and is not the idea I've gotten out of our general agile meetings. In the positive case, QA sets the issue to done anyways. Unless the reason for this is to actively enforce us to improve on your first two points (mainly tests).

Write documentation for the surrounding context or the feature

This is a requirement to pass the review, so it should be done before a review is requested.

[HendrikThePendric]

We use Components to indicate which parts of the system is affected by an issue. One issue can affect multiple parts of the system, e.g. both frontend and backend for an application.

Today we noticed that having one issue (i.e. DHIS2-169) that affects multiple components is often not completed for all. Maybe we could add some instructions here for dealing with these cases. Something like this: Issues that contain references to multiple components should never make the "In progress" stage in their current form. Instead they should be split up into N issues (where N = ComponentCount).

[varl]

There are times where a developer must wait for something, e.g. for a code review, for testing, etc. It is tempting to start on a new ticket in JIRA to avoid idle time.

Are we considering these as 'in progress'? In my mind we pass 'the torch' of the issue with these statuses, especially testing.

I consider the PRs in GitHub as "In progress" in JIRA. The "Review" status in JIRA is a final verification that:

• There is a Zephyr test case linked
• That the test case can be executed successfully
• The JIRA issue has correct fix versions
• All code has been merged to the relevant branches
• The user documentation has been updated appropriately (linked PR to dhis2-docs)

This shouldn't take that long, and if it does we need to talk about it in @dhis2/team-apps.

So I tend to start looking into a new issue once an issue I'm working on is set to testing. I understand that I am the issue owner, and therefore overall responsible for the issue, and will follow it up if it doesn't pass testing (or review).

The risk is that when something new is being worked on, and something bounces back from testing, it falls between the cracks. We have issues that have bounced back from testing 6 months ago that have not been revisited and fixed.

Sometimes it takes a considerable amount of time before the tests are executed, and waiting for these to pass until I can start a new issue seems unecessary,

There is flexibility here, and if you feel that the context switch isn't a problem, go ahead and start on a new issue. Consider the rule that you only have one issue in the "In progress" column at a time. If you have one in "Review", then you can have another "In progress".

and is not the idea I've gotten out of our general agile meetings. In the positive case, QA sets the issue to done anyways. Unless the reason for this is to actively enforce us to improve on your first two points (mainly tests).

It's not about enforcement, it's about quality and getting issues done-done. Loose ends are bad as they continue to fray they lead to broken windows. It doesn't just apply to the code, it applies to JIRA as well.

Write documentation for the surrounding context or the feature

This is a requirement to pass the review, so it should be done before a review is requested.

Definitely!

In addition to that, we have other forms of documentation that is not part of the review process, e.g. developer facing docs for libraries and apps: e.g. maintenance docs/, JSDoc, dev facing docs, etc.

[varl]

Today we noticed that having one issue (i.e. DHIS2-169) that affects multiple components is often not completed for all. Maybe we could add some instructions here for dealing with these cases.

Done! 👍

[Mohammer5]

So I tend to start looking into a new issue once an issue I'm working on is set to testing.

@Birkbjo To give you a bit of context here, I was talking to @varl yesterday, saying that I feel a bit frustrated with having too many context switches, hence we were talking about what I/we could do to reduce exhaustive context switches

To me, this doesn't start with testing. I often found myself starting a new topic when requesting the review for the main PR, delaying backports to when the PR has been accepted, because some have a lot of changes and it's almost guaranteed that I have to adjust things. In the worst cases, this can take weeks.

I don't think working on two issues (one in "In progress" and one in "In review") is too much of context switching normally. It can happen though that you end up with two or three issues in "In review" and one in "In progress", and whenever something changes, some of the questions I ask myself when I get a notification that something has been updated are:

• Is it a critical bug that has much higher priority so I should stop working on what I'm doing now
• Do I have enough time to finish both what I'm working on now and the updated issue so they make it into the next minor release
• Is someone waiting for my PR to be merged / backported, do I silently block someone else work or should I respond to the update simply because it's quite a cognitive overhead to read into the code changes and I don't want that person to wait too long for my response, otherwise they'd have to read all that again
• etc.

Then I'm already trying to think about both issues, that might be a personal issue which you guys don't have, or maybe you do. So of course it's up to us, how many issues we can handle once they're in "In review", but with what @varl has added, we now have to possibility to not start a new JIRA issue if we feel it'd interrupt our workflow too much and we use the time holes instead to either do some refactoring, increase documentation/test coverage or work on one of our libraries because the switch doesn't trigger as many administrative questions)

Write documentation for the surrounding context or the feature

This is a requirement to pass the review, so it should be done before a review is requested.

The surrounding context doesn't need to be the context of the feature. When you realize that there's documentation missing that could've helped you working on the issue but has nothing to do with the issue itself, you're not required to document that part of the app

[varl]

Migrating this to a document: github.com/dhis2/notes/blob/master/platform/processes/platform-fe-jira.md