Centralized Issue management in single separate repository in kyma-project org

[Ressetkk]

Created on 2023-05-29 by @Ressetkk.

Decision log

Name	Description
Title	Centralized Issue management in single separate repository in kyma-project org
Due date	-
Status	Proposed on 2023-05-29, Rejected on 2023-06-13
Decision type	Binary
Affected decisions	#613 (for the record)

Context

This proposal will apply to all kyma-project repositories!

Modularization topic brought a lot of changes in available module management options. I see it as an opportunity for increasing the security of the repositories and increase the ability introduce more advanced Git repository lifecycle management. This proposal is meant to set-up a standard that lets deal with issues in a way that developers have been gotten used to, but allows to harden the security and enable automations and PR lifecycle based on consuming GitHub events and letting external system deal with lifecycle, so automation offerings are extended.

Motive of this proposal

Currently kyma-project relies a lot on integrated GitHub configurations. From personal point of view as someone who manages GitHub accesses and permissions across organization, also as a person who is responsible for providing convenient automations for developers to use in their repositories I have to say that GitHub currently allows very basic functionality for automations. Since all members conveniently use native GitHub features involving metadata management that requires write access to the repository. This itself requires to force GitHub limited security functionalities without ability to configure it to general needs.

This functionality can be changed and secured by moving all options that require active write access to edit and modify issue metadata, comments, titles and descriptions into separate repository containing all issues. All developers would have write access to that repository, which ensures GitHub edit capabilities are working to the fullest. This includes adding labelling automations, as present in test-infra. area/ labels can be reused or adopted so they reflect the module affected by the issue. With that it'd be possible to pin single issue to multiple modules if it's applicable.

I expect that with well established standard keeping a single board for increasing number of modules will benefit entire organization management in the long-term.

Limitation of current workflow

• Code Ownership gives lack of support for unattended auto merge automations without significant workarounds like adding bot user as member of GitHub team, creating multiple users that approve PRs for each other, creating fake statuses to meet GitHub's branch protection requirements
• Every developer needs write access to any repository in kyma-project repo to edit issues, even if they don't need to have it
• Any member in organization can push branches and tags to any repository, which may create disruptions in already configured jobs/pipelines created by teams and makes significant maintenance effort

Decision

Decision is to move all active and open Issues from separate repositories into single bugtracker or issues repository with following requirements changes:

• Every issue contains area/ and kind/ label after transfer
• area/ labels reflect the module the issue touches
• There is an automation that makes sure issue contains required labels (eg. area/, kind/, optionally triage/ priority/), otherwise Issue author will be notified through GitHub comments
• Update Issue templates to reflect new required options (possibly with help of technical writers)

Consequences

Pros

• Module repositories can have Issues tab disabled
• Single repository containing all issues from infinite number of separate modules
• Access for the developers to module repositories can be restricted only to number of people or given to automation completely
• Boards management should be more consistent - common requirements for labeling, name conventions, other rules, etc.
• Issue templates are managed only in single place. Can be easily modified, updated without the need to propagate them to large number of repos

Cons

• Linking issues to PRs with GitHub keywords will not be possible (tbc)
• All modules will have to include the link to new issue creation in separate repo in their README.md
• It's a standard that everyone in kyma-project will have to follow, no matter the module configuration
• One repository will have hundreds of issues (major drawback)

[pbochynski]

I see kyma-project/kyma repository as an entry point and a default place to create an issue, but only for those who do not know the right place. And in my opinion, the right place is where the code is located. Module maintainers would prefer to keep their backlog in the repository of their modules. The issues created in the main kyma after triage should also land in the target module repository. This way issue management should be easier and each team will deal only with a small number of issues and can keep them properly maintained (prioritized, well described, and finally resolved and closed).

Another aspect touched on in the proposal is excessive permissions. Moving into module repositories will reduce that problem. Only maintainers should have write access. I do not see the need to revoke write access from maintainers. I as a manner and person responsible for the module want to have the possibility to bypass and overrule bots (or broken automation). Another aspect is how we manage other maintainers (like technical writers). They usually get write access, but in my opinion 'triage' or even read would be sufficient.

Short summary:

• I am against keeping all issues in one repo - it is against modularization
• with dedicated repositories for modules we can limit write access only to the maintainers
• consider using triage permission instead of write to keep the possibility to manage issues without having write access to the repository.

[Ressetkk]

@pbochynski I well understand your point, however there is a specific problem with GitHub permission system - no person can be code owner without explicit write access to the repository. That means triage role cannot be used for people that need to be code owners.

Let's have an example - you hire a new student and you want to give them some permissions to assign him to review. Currently you'd have to give them full write access to the repository, because if you add them to the team in settings, they can't be assigned automatically, because they are not owners. And giving them full write access without proper grace period may lead to some problems.

Additionally, given some of the labels on PRs and issues are used for PR lifecycle, any write access to the repository with code can result in somebody altering the entire lifecycle of PR and may cause an unintentional merge of a PR. Maintaining no physical access to the repository,

I expect that if we manage to keep issue management as standardized single board, all teams can have their workflow organized through the GitHub boards the same way as it is right now. The effort for the teams would be the same, we'd be able to keep some degree of standard across entire organization, possibly we'll have less amount of duplicated issues, we'd be able to maintain and keep issue templates and automations in single place, and most importantly - finally we will be able to restrict access to the repos with the code, so some degree of desired automations by teams will be achievable without effort.

[pbochynski]

Let's have an example - you hire a new student and you want to give them some permissions to assign him to review.

I think students should start with a few contributions before they do their first review. I do not see it as a valid use case. I still think with polyrepo approach we can use native github features and assign write permissions only to true codeowners.

[uschulle18]

Didn't we have a somehow similar proposal in the past which led to a lot of frustration of our developers? So much, that we had to reverse it. From very high level I would really not decrease writing rights of devs if not strictly necessary.

[valentinvieriu]

I think we just moved away from the one repo for all modules. I agree with @pbochynski that the issues/feature requests should be as close as possible to the code. We use Github Projects to capture all issues from various repositories together. I am against this proposal. I agree with @pbochynski that we should have an overarching capture-all repository, where people can create issues but then can be closed and moved to the proper repository.

[Ressetkk]

Let's have an example - you hire a new student and you want to give them some permissions to assign him to review.

I think students should start with a few contributions before they do their first review. I do not see it as a valid use case. I still think with polyrepo approach we can use native github features and assign write permissions only to true codeowners.

Unforunately, this is not the case. You add new hire as a team member, they automatically get write permission, are assigned to review and can modify everything. We don't have any control over it as organization admins.

Didn't we have a somehow similar proposal in the past which led to a lot of frustration of our developers? So much, that we had to reverse it. From very high level I would really not decrease writing rights of devs if not strictly necessary.

There was a solution to move away from GitHub limited support, but the issues were not very accessible through boards, as devs couldn't edit issue metadata. This proposal actually pushes the issues to separate repo where everyone could have write access and pick up the issue onto their boards, which will not change. With that, GitHub admins will have full control over securing code and access to it, we will be able to provide quality gates and automations in GitHub in easy and secure way.

I think we just moved away from the one repo for all modules. I agree with @pbochynski that the issues/feature requests should be as close as possible to the code. We use Github Projects to capture all issues from various repositories together. I am against this proposal. I agree with @pbochynski that we should have an overarching capture-all repository, where people can create issues but then can be closed and moved to the proper repository.

Team boards will stay the same, code will be separated, onto thing that would be common is the issue board.

---

I just would like to give the accessibility for the admins to actually have control over adding additional custom blocking requirements and reduce the overhead of maintaining issue-related automations. With requests that sometimes come up to Neighbors, like unattended merges of PRs, direct pushes by bots, additional blocking governance checks across repos, we can't do much with current setup.

Labels are source of truth for determining Issue/PR lifecycle - if anyone can change labels you can't really use it as one. With that we can't even extend the PR lifecycle and block it from merging if some quality checks are not met (eg. missing labels, release management, etc.). Take a look at how gardener managed to do their workflow.

I understand all concerns. I'm all for not altering write access but if you can accept that code may end up being less secure, as entire teams would have full write access to their repos then let's reject this one.

[k15r]

Unforunately, this is not the case. You add new hire as a team member, they automatically get write permission, are assigned to review and can modify everything. We don't have any control over it as organization admins.

honestly i do not see the problem here: even with write permissions you cannot bypass the review requirements. Automatic assignment to reviews can be setup in a way that it does not assign certain people.

But back to the initial proposal: moving all issues to one central repo. I am not a big fan of this idea. Imho if we would want to move issue management to some central location, then we should directly use a different tool and only handle the prs using github. but as long as we do want to keep our issue management on github, then we should keep it in the same location as the code. Because as much as i dislike github for issues it has one major advantage over external tools: it is super convenient to use. We would lose this convenience with the proposed solution and not get real benefits for the daily work.

[Ressetkk]

honestly i do not see the problem here: even with write permissions you cannot bypass the review requirements. Automatic assignment to reviews can be setup in a way that it does not assign certain people.

Automatic assignment restriction still doesn't filter out their approving review. If they are not assigned, but belong to the group with write permission and code ownership, their approving review is as valid as everyone else. GitHub doesn't give you the ability to conveniently restrict approvers and reviewers, so the work can be distributed onto people doing review and actual people approving PRs. I really just want to enable us to intercept and improve our PR lifecycle flow in a secure way. Without proper permission system and agreements we may never achieve it.

[pbochynski]

Unforunately, this is not the case. You add new hire as a team member, they automatically get write permission, are assigned to review and can modify everything.

@Ressetkk I disagree. We have a clear guideline on how to become a code oner](https://github.com/kyma-project/community/blob/main/docs/governance/01-governance.md#become-a-code-owner). If you add a new person that never seen the code before as a maintainer you are doing something wrong. I suppose your assumption is coming from the fact that we have a lot of GitHub teams that match real teams (both names and members), but in my opinion, we should rather use teams to group people with the same permissions in the same area. Good example is team observability that groups Maintainers of monitoring, logging, tracing and telemetry module.

[Ressetkk]

Then this also requires some more visibility. I feel like the line between team and repo maintainer is so blurred that no one really cares about it. Current structure of permissions does not reflect this kind of state I think. Code ownership is distributed between teams and if anyone new comes into the team, they get assigned the GitHub membership, and automatically have the same permission as others. The point is - GitHub permission system is just too basic to even support more advanced use-cases. But we are going a bit OT here.

[Ressetkk]

I was looking for enablement for PR lifecycle services improvements. It may severely touch GitHub permissions. That proposal was one of the ideas to provide enablement without affecting user experience too much - it all goes down to one repo where everyone will have write permission, separate repos with code where PR lifecycle will be managed by bot incl. specific users with write access. And as I said, current settings will prevent us from extending this lifecycle with those improvements.

But I rather feel the discussion can be already closed. Looks like more people are against it so let's not waste time on this.

[pbochynski]

The proposal didn't get support from the team. Closing as the author suggests in the last comment with the status Rejected