Document the Workflow

[rochacbruno]

Workflow

@omkarkhatavkar @flatfender

I would like to discuss here the workflow and then this will be transformed in to a docs/workflow.md with all the details.

Given {source} as (redmine/github/anyplugin) we can assume that:

Issues

1. New unassigned issue is CREATED on {source} ACTION: no-op Jirasync should ignore this as it is not assigned and we don't want to create backlog for new issues that are not ensured to be included in a sprint. (some issues are opened and then never worked on, and some issues are opened as meta-issues)

   Current results:

   • Github: NOT OK Jirasync is creating new task for that issues and should not
   • Redmine: OK unassigned issues not synced

2. New assigned issue is CREATED on {source} ACTION: create Jirasync will read the data and create a new task on Jira for this issue and will assign it to the related user. (reporter will be the author of the issue)

   Current results:

   • Github: NOT OK Jirasync is not fetching this event, so the issue is not being created for the assigned user.
   • Redmine: OK Assigned issues in redmine it is synced

3. An existing issue is ASSIGNED on {source} ACTION: create Jirasync will read the data and create a new task on Jira for this issue and will assign it to the related user. (reporter will be the author of the issue)

   Current results:

   • Github: NOTOK Jirasync is not fetching this event, so the issue is not being created for the assigned user.
   • Redmine: OK Once issue is assigned on Redmine it is synced

4. An existing issue is UPDATED on {source} ACTION: update Jirasync will update the existing task on jira keeping status, assignee, title fields synced.

   Current results:

   • Github: Jirasync is keeping status updated but missing assignee and title
   • Redmine: Jirasync is keeping status and assignee and title

5. An existing issue is CLOSED on {source} ACTION: complete Jirasync will mark task on jira as Done Current results: Github/Redmine: OK Jirasync is marking as done.

6. An existing issue is Reopened on {source} ACTION: Reopen Jirasync will mark task on jira as Reopen (back to backlog of user) Current results:

   • Github: Unknown
   • Redmine: OK Jirasync is marking as Reopened.

Pull Requests

Applies only for Github {source}

NOTE: I have not tested all the Pull Request workflows yet, will do and provinde feedback here, I need to confirm the expected results.

1. New PR CREATED ACTION: create Jirasync will read the data and create a new task on Jira for this PR and will assign it to the PR Author and also move the task to the current sprint Current results: OK Jirasync is creating a new task for PR author

2. PR is ASSIGNED to a different user ACTION: create Jirasync will read the data and create a new task on Jira for this PR and will assign it to the PR assignee and also move the task to the current sprint Current status: NOT OK Jirasync is ignoring the assignee change

3. A PR Review is REQUESTED ACTION: no-op Jirasync does nothing with that info until a review is really made Current Results: OK Jirasync is ignoring it

4. A PR is REVIEWED by user ACTION: create Jirasync will read the data and create a new task and assign to the reviewer and also move the task to the current sprint Current Status: OK issue is created as pr_review for reviewer

5. Existing PR is UPDATED (title and other fields change) ACTION: update Jirasync will update the title and other relevant fields on existing Jira task (also dependent tasks will be updated, like the pr_review tasks) Current Status: NOT OK Jirasync is fetching only status changes

6. PR is CLOSED or merged ACTION: complete Jirasync will mark all the tasks related to this PR (author, assignee, reviewers) as Done Current status: OK Jirasync is marking pr_create and pr_review as done

[omkarkhatavkar]

@rochacbruno Excellent Job putting all here in an issue.

The new unassigned issue is CREATED on {source}

• Github: NOT OK Jirasync is creating a new task for that issues and should not ==> We should not create Task in backlog unless it was assigned to anyone. I totally agree on this point, I think we add more validation in case if that issue is assigned to any team member from a valid list of users in that case only it will be sync to JIRA. If there are multiple users are assigned to an issue, then it will difficult to identify the correct user. This workflow will then need to be handled manually.

The new assigned issue is CREATED on {source} ==>We are having the function to fetch assigned issue, I have added a check if we have multiple peoples are assigned to that issue then don't fetch. I will remove that check we could see all signed issues. But the challenge we can not update issue assignee each time for each people hence we can stop updating assignee and then it's people responsibility update, correct assignee

title-change => We can create an issue for this. I see this is a certain corner case, we will work on this resolve this. This is a valid issue but will work on the basis of priority.

PR is ASSIGNED to a different user => This is also a corner case where people won't assign their self PR to Others. It's a valid issue, less priority. (my thought)

[flatfender]

Couple of Questions.

1. How do we handle syncing at the end of a sprint? I'm not 100% sure I agree with aligning it to a sprint as it's the responsibility of the jira card holder to plan the sprint based on workload. I do see where the Jira tasks should be completed in the current sprint in theory, but what about when the sprint is in the last day or two and the Jira user will not have time to complete during the active sprint? Now we've messed up our sprint metrics by including cards. For this reason, i'm partially against putting things into the sprint.
2. Are we doing any mapping of Components? Components are required and I have a concern if we don't map the Components users will forget to map them as they didn't create the cards.

[rochacbruno]

@flatfender

1. Good question, I suggest we remove (for now) all the code that deals with sprints and let this responsability for the card holder

2. Currently Component defaults to Automation but we can include a better mapping based on labels or static names.

[omkarkhatavkar]

@flatfender :

1. Currently, people who are worked on PR_Create, PR_Review those are things are put in the current sprint. Consider this example that people have worked on something then it getting putting into Sprint, unlike Bugzilla where people chose on which Bugzilla where they want to work. (putting in current sprint recording people's effort immediate basis ). At the end of the sprint non-complete will be moved manually anytime.

One another thing is that if we put everything into backlog, then it will be a little messy for people to choose and select then put in the current sprint. (there will be a bunch of them). I think it will be good for them to see the only non-complete task at the end of the sprint.

All assigned github issues will be part of the backlog, those are not put into the current sprint. Pr are more active hence one reason to put in current sprint.

2. As @rochacbruno said this has been taken care, we will not have a ticket without components

[sghai]

Couple of Questions.

1. How do we handle syncing at the end of a sprint? I'm not 100% sure I agree with aligning it to a sprint as it's the responsibility of the jira card holder to plan the sprint based on workload. I do see where the Jira tasks should be completed in the current sprint in theory, but what about when the sprint is in the last day or two and the Jira user will not have time to complete during the active sprint? Now we've messed up our sprint metrics by including cards. For this reason, i'm partially against putting things into the sprint.

Valid point Matt as it might affect sprint metrics. I feel we are using jirasync mostly to track automation efforts so its important to keep alignment with sprint. Although it can be done manually by card holder to put into sprint but the real problem, i see, is finding the card from a big backlog. I might be wrong.

on other side, when we will trigger the sync with Jira ? what would be the cadence ? What if we trigger/schedule the sync after a week(mid of sprint) considering 2 weeks of sprint ? If we are doing sync at end of sprint then cards should move to next sprint ? what do you think ?

2. Are we doing any mapping of Components? Components are required and I have a concern if we don't map the Components users will forget to map them as they didn't create the cards.

Only one component needs to be mapped and i.e. 'Automation' and we should do it.

[sghai]

@flatfender :

1. Currently, people who are worked on PR_Create, PR_Review those are things are put in the current sprint. Consider this example that people have worked on something then it getting putting into Sprint, unlike Bugzilla where people chose on which Bugzilla where they want to work. (putting in current sprint recording people's effort immediate basis ). At the end of the sprint non-complete will be moved manually anytime.

One another thing is that if we put everything into backlog, then it will be a little messy for people to choose and select then put in the current sprint. (there will be a bunch of them). I think it will be good for them to see the only non-complete task at the end of the sprint.

All assigned github issues will be part of the backlog, those are not put into the current sprint. Pr are more active hence one reason to put in current sprint.

@omkarkhatavkar valid point. We need to think about when we want to schedule the sync and its cadence first. If its end of sprint, them metrics would affect if we keep all cards in same sprint as Matt pointed and we want to avoid that. See my comments here https://github.com/omkarkhatavkar/jirasync/issues/15#issuecomment-518933327

1. As @rochacbruno said this has been taken care, we will not have a ticket without components

[omkarkhatavkar]

We could use some time frame where we can sync the data. We could easily avoid running 2-3 days before the closing of the sprint. By this way, we will avoid the added task in the sprint on last moment. This way we will not make any damage to metrics generated using JIRA Sprint.

[rochacbruno]

We could use some time frame where we can sync the data. We could easily avoid running 2-3 days before the closing of the sprint. By this way, we will avoid the added task in the sprint on last moment. This way we will not make any damage to metrics generated using JIRA Sprint.

But what happens with the issues/PRs that are updated/closed on that 2-3 days interval?

It will not update the status on Jira and for the new issues users will tend to create manually on Jira which will end with duplicates on next run.

I vote for not dealing with sprints and keeping this as a user responsibility, there are some ways to search Jira tasks ordered by date of creation and naming pattern, also you can do actions in bulk moving a bunch to a sprint in a single click.

[rochacbruno]

@sghai

, when we will trigger the sync with Jira ? what would be the cadence ?

We are thinking about a running daemon which has a loop with 5 minutes interval, so that means that the sync must happen on interval 5 minutes + time taken to run the script ~2 minutes. So every 7 minutes we are going to sync.

[omkarkhatavkar]

@rochacbruno @flatfender @sghai I think we need to find the solution such that It will be very less manual effort and does not affect the sprint metrics. Putting everything in back_log then manually putting everything in Sprint will be overburden for people. This is not a single day attempt, this need to be done every day. I would recommend this will be hectic for people. I think it's a more manual thing.

Even if people added a task in the sprint, People will do not have control cover completing their PR create task or review task to be in the current sprint because it is very interdependent other people.

Likely this will happen that any task create later in the sprint has fewer chances to get complete.

Solutions from my side:

1. As we know sprint is made of 2 weeks ultimately 15 Days. We can only sync till 10 Days then next run will run in next sprint. Whatever task has been created in between buffer will be sync in next sprint anyways. (as interval can be changed) . We will not have duplicates anyways because there is a check of task

2. We can add the option of skipping create operations. By this way, only tasks which are created in the first week of the sprint (time can be decided) will get updated entire sprint and buffer tasks will be sync in next sprint.

Note: We should not have any duplicate task as any task which is completed any respective sprint taken care and no Duplicates will be created.

[rochacbruno]

@omkarkhatavkar @flatfender @sghai

To be able to move forward with deploying this project I suggest:

1. Lets separate this project in phases Beta
   • Manual deploy to some QE box or Openstack instance
   • Do not deal with sprints (users will need to do manual action for sprints).
   • All the features of the above workflow must be fully working.
   • Run it as a daemon in cycles of 5 minutes.
   • We will do the monitoring of the service manually, will check status, logs etc once a day/week
   • We will gather feedback in form of issues/bug reports/RFCs
   • Milestone: Deploy phase 1 it by the end of next week (Aug 16) if we have fixed the credentials problem or ASAP GA:
   • After 3 weeks running in phase 1 we move forward
   • Analyse the bugs failures etc and see what we can improve/fix
   • Based on feedback see what we can automate related to sprints and overall workflow
   • Adopt an automated monitoring solution
   • Automate the deployment
   • Milestone: End of September

This way we can move forward and will enable us to see the usability problems in practice instead of trying to predict it.

NOTES:

Once deployed we will send an announce to users explaining the beta phase, explaining that the tool is not 100% reliable yet as it is a testing phase and asking them for help to keep :eyes: open and report if some issue had sync errors or have not been synced, during the beta users will need to check it for us (let them work as QA for us) :smile:

[bherrin3]

As an outsider looking in, just some feedback:

Clarity

0. On the project README.md, it might be worth having a small table for initialisms, just to ensure vernacular is clear. It never hurts. This may be implied, but just to be explicit in feedback.

1. I would ensure the specific working {sources} are explicitly listed in the README.md. While it is good with the combination of [0] it keeps the implemented purpose clear.

Functionality Questions

1. Sprints -- there are a lot of variables here and a lot of "how your team works", philosophies of Sprints, etc. Therefore, it would be best to NOT put new items into a Sprint.
   • assignees don't (typically) control when a sprint begins or ends.

Jirasync should ignore this as it is not assigned and we don't want to create backlog for new issues that are not ensured to be included in a sprint

I agree with @rochacbruno 's method of collect data and let's see.

It is easy for the user to check out their backlog. I am not sure if I am missing some context, but this should never be a problem (as all of these items are coming in assigned):

Backlogs should not be unbound. They should be reachable in the next 1-3 sprints or off your backlog (you are overloaded or the issue actually isn't that important).

2. Are the source issues referenced in the JIRA tickets? I may have missed where this was included. This may just be another Clarity issue.

3. I saw a note about this, but I wanted to ensure that a check is always done to ensure the JIRA issue hasn't already been created.

We will not have duplicates anyways because there is a check of task

4. It may be worth exploring a fixed component mapping from the incoming issues to our current component definitions. I can see instances where Infrastructure, Testing, Management, and Automation have been relevant to Redmine issues.

It may also be worth considering mapping the {source} as a component as well. i.e. pulp2 = [ pulp_docker, pulp_file, pulp_rpm, etc] , pulp3 = [ "other p3 repos" ], etc

Thanks, Rocha!

Great work, @rochacbruno . Let me know if any of the feedback needs clarification.

[omkarkhatavkar]

@bherrin3 Thanks for the feedback,

Clarity

1. On the project README.md, it might be worth having a small table for initialisms, just to ensure vernacular is clear. It never hurts. This may be implied, but just to be explicit in feedback.
2. I would ensure the specific working {sources} are explicitly listed in the README.md. While it is good with the combination of [0] it keeps the implemented purpose clear.

It is a really good point, There was a lot of changes in commands and processes, we have to update that in the readme.md file. This will take little time, we will ensure that it will be more clear and crisp.

Functionality Questions

1. Sprints -- there are a lot of variables here and a lot of "how your team works", philosophies of Sprints, etc. Therefore, it would be best to NOT put new items into a Sprint.

• assignees don't (typically) control when a sprint begins or ends.

Jirasync should ignore this as it is not assigned and we don't want to create backlog for new issues that are not ensured to be included in a sprint

I agree with @rochacbruno 's method of collect data and let's see.

It is easy for the user to check out their backlog. I am not sure if I am missing some context, but this should never be a problem (as all of these items are coming in assigned):

My thought process was pr_create, pr_reviews are showing that there was work was done on something, So why not putting current sprint. Although I do agree on sprint metrics questions where we do not want those to get affected. Everyday user does need to visit there Jira own backlog and do need to care to put them into a sprint or not. There could be a case where a person did not update the Jira ticket into a sprint and within some time if that PR got merged. Ticket status automatically will get updated to done without assigning to the sprint.

If still want to get people to feedback I'm welcome on that.

2. I saw a note about this, but I wanted to ensure that a check is always done to ensure the JIRA issue hasn't already been created. We will not have duplicates anyways because there is a check of task

3. It may be worth exploring a fixed component mapping from the incoming issues to our current component definitions. I can see instances where Infrastructure, Testing, Management, and Automation have been relevant to Redmine issues.

This is a really good point if we have something GitHub or Redmine helping us to know that a particular task can be categorized, we can do it easily. I see this as a little challenging, @rochacbruno we have to create an issue for this?

It may also be worth considering mapping the {source} as a component as well. i.e. pulp2 = [ pulp_docker, pulp_file, pulp_rpm, etc] , pulp3 = [ "other p3 repos" ], etc

We can map multiple repos, not sure if that question was about that.

[rochacbruno]

It may be worth exploring a fixed component mapping from the incoming issues to our current component definitions. I can see instances where Infrastructure, Testing, Management, and Automation have been relevant to Redmine issues. This is a really good point if we have something GitHub or Redmine helping us to know that a particular task can be categorized, we can do it easily. I see this as a little challenging, @rochacbruno we have to create an issue for this?

We can in future (after beta) explore the idea of using github labels and redmine project as component/labels.