[FEATURE] OpenSearch-Project workflow manager

peternied commented 1 year ago

Is your feature request related to a problem?

As an organization, there are policies that should be enabled on all repositories, such as enforcing the use of the DCO [1], supporting consistent platforms in continuous integration tests [2], and ensuring repositories are using consistent labeling for untriaged issues [3].

There are easy ways to roll out these changes, such as the GitHub App that has been used for DCO checks. Or the more often-used creation of a campaign with issues created in all repositories, actively driving those issues to resolution in all repositories.

GitHub Apps need to be a special purpose and take effort to create, publish, and maintain. Whereas manual issues can support any kind of flexibility.

Maybe there is space for a middle ground that could be easier to centrally manage and support customization.

What solution would you like?

Let's build a system to can inspect and update GitHub Action workflows in the organization. Built out as a GitHub Application it would read from a repository to discover workflows and then deploy to those repositories. Add some filtering metadata on those workflows and different policies would be applied where needed.

This would allow authoring a workflow update/create, review, and then deployment to be managed via a single pull request in the configuration repository. Adding individual workflow is simpler than a GitHub App and easier to customize based on the target repository.

Do you have any additional context?

dblock commented 1 year ago

I feel like this would be a step backwards, as the apps significantly simplify deployment and maintenance, and are automatically enabled for new repos.

peternied commented 1 year ago

For me, the overhead for an individual app is prohibitive for handling the untriaged label. Here is a diagram that might help clarify this proposal for a GitHub App that can be configured/managed via an open centralized config repository - does this still seem like a backwards step?

graph LR
   app(Workflow Management<br>GitHub App) -- Install App for all repos --> osp(OpenSearch Project)
   app -- Read Workflows / Config --> cr[GitHub Org Workflow Configure Repo]
   app -- Sync managed<br> .github/workflows/* --> sec[Security Repo]
   sec --> gha(Github Actions)
   app -- Sync managed<br> .github/workflows/* --> os[OpenSearch Repo]
   os --> gha
   app -- Sync managed<br> .github/workflows/* --> etc[[... Repo]]
   etc --> gha

style gha fill:#9f6,stroke:#333,stroke-width:4px;
style app fill:#f96,stroke:#333,stroke-width:4px;

Would you prefer we made a GitHub app for each kind of workflow we'd like to enforce, such as the untriaged flow?

dblock commented 1 year ago

For me, the overhead for an individual app is prohibitive for handling the untriaged label. Here is a diagram that might help clarify this proposal for a GitHub App that can be configured/managed via an open centralized config repository - does this still seem like a backwards step?

I think so, but it's just my 0.02c because of below ...

Would you prefer we made a GitHub app for each kind of workflow we'd like to enforce, such as the untriaged flow?

Yes. Conceptually org-wide processes should be enforced at org-wide level. I actually have more feature requests for the label app. For example, I'd like to enforce a minimum set of labels across all projects, such as performance.

dblock commented 1 year ago

I think we need something like https://organizer.gitconsensus.com/#/

peternied commented 1 year ago

Nice find! I'd much rather adopt an existing tool and we can always contribute back if we'd need more functionality. Want to create an issue with a proposal for it / your use cases?

dblock commented 1 year ago

I opened https://github.com/opensearch-project/.github/issues/115

saratvemulapalli commented 1 year ago

Coming from https://github.com/opensearch-project/opensearch-plugins/issues/201#issuecomment-1429020772 This is nice!

peternied commented 1 year ago

github-org-sync(-action) is a tool to sync files from a skeleton/template repository to other repositories in your organization.

Found github-organization-sync-er while that tool doesn't look like it is in active development, maybe it could be resurrected or forked?

opensearch-project / .github