sparshev
commented
2 years ago With Aquarium we're building the environments which can be completely sandboxed and that's taking the security issue of "what's executed on them" not so related anymore. For sure there is still some concerns about the generated artifacts, but in general noone but developers now controls the pipelines and tools they are using.
I can identify a couple of things need to be taking care of:
- Scheduler and orchestrator - should be simple without much functionality, just bare way to create the graph, choose the environment (label) for each task and make it running. The additional features could be added later but should not take care of the environment components.
- Verifiability - the actions need to be verifiable to prepare a proper report of what the code was executed during the pipeline and to show the signatures of operations to compare with approved ones for release pipelines. On general PR it's not much important (could generate warnings but pass to allow to decide during the review process), but for release ones there should be a way to block such artifacts from being released.
Keeping that in mind I think will bring the CI/Release pipelines to the new era of automation and flexibility.
The Aquarium cluster can actually become a useful serverless platform due to it's distributed nature. For now we're using jenkins, but it's incapable of HA due to file-bonded database. Need to think about how it could be possible to run jenkinsfilerunner or some another scheduler of pipelines.