Making new workflows propagation safe?

[f1ames]

Issue

Since we are integrating setup-workflows workflow with our most used repos (already added in ckeditor4, ckeditor4-presets, ckeditor4-angular, ckeditor4-react, ckeditor4-vue) I started to wonder how to make adding new workflows here (which then will be auto-propagated) safe.

This is a repo for common workflows in fact so the assumption is that repo-specific workflows will not land here. However:

• Some workflows may break something in one (or more) repos due to some assumptions made only for specific repos.
• There might be some configuration/file/labels missing, breaking the workflow, which will require to be added manually in each repo (and since we automated workflow propagation we don't want to do this :smile: :see_no_evil:).
• Some repos (e.g. one out of ten) may not need specific workflow.

Ideas

New workflows should be disabled by default

We can have new workflows to be disabled by default. To enable them one would need to change some config. It will prevent from uncontrolled propagation but requires manual work. On of the main purposes of this repo is to automate as much stuff as possible.

Have strong prerequisites validation policy

This means the workflow itself should validate if it can be run on a given repo and clearly communicate if something is missing. This would be done per case - e.g. check if specific config / issue label / npm command / etc is there and only run if all the requirements are fulfilled.

Add setup step

Each new workflow when added could try to setup all required stuff to run smoothly later (this looks like quite complex solution on the first sight).

Add workflows configuration

Add configuration file with some default values for workflows (branch names, label names, execution time or even flag saying that workflow is disabled by default).

The above are not inclusive and we may thing of implementing more than one approach. WDYT @ckeditor/ckeditor-4-core-developers?

[Dumluregn]

Trying to setup everything would probably consume most of the time we save up on the given solution, and the necessity of turning the workflows manually in each repo is very error-prone (easy to omit some of them etc.). So for me the best "benefit-cost ratio" has the Have strong prerequisites validation policy idea.

[jacekbogdanski]

Do you have any repo in mind currently? I know it's sometimes good to predict issues in advance, but if we don't have any real case to solve right now, we may be missing the context. As an example, if we will have only single repo requiring some action, this action should rather go to the repo directly. On the other hand, if 9/10 repos are fine with some action, maybe this single repo shouldn't be automated at all?

BTW we could also just use different repositories for some customizations. The whole concept of this automation was not doing anything manually whether possible, so additional config options, environment variables, etc are staying in opposite to the idea.

[f1ames]

Do you have any repo in mind currently?

Currently we have two cases:

• CKEditor 4 with branch protection rules (one can't push to master/major branch without review) - #6. This maybe could be solved by changing GH bot user privilegies (but if it requires admin it's no-go I suppose).
• Angular integration with specific package.json file placement - #5.

So this are the real world cases. Ofc, there are multiple solutions and also we are trying to keep repos structure and workflows as consistent as possible, but sometimes it's not possible. Adding ability to slightly adjust workflows or get valid information or stop workflow execution for such cases could be useful.

Hmmm, the above were reasons for reporting #4 mostly. So for now, we don't have a case (fortunately) where workflow was breaking something else. And since that is the main idea behind this ticket we can wait and see. But still it will be good to keep in mind when introducing new workflows.