What is the role of CI/CD (and Tekton) in a developer's local workflow?

bobcatfish commented 4 years ago

In https://github.com/tektoncd/pipeline/pull/2903 @vdemeester and I started discussing the role of Makefiles when using a Tekton based CI/CD workflow, and then we started talking about the potential differences between what CI/CD runs and what an engineer runs as part of their developer workflow.

I believe that being part of a developer's local workflow is in line with Tekton's mission but @vdemeester disagrees.

I'd like to discuss this further and establish:

If it is part of Tekton's mission
And if (1), what do we need to change in the mission to reflect this

bobcatfish commented 4 years ago

I've started this doc with my thoughts: CI/CD and the development workflow - not sure the best medium to discuss this, but starting a doc seemed like a good starting place? Open to other ideas!

vdemeester commented 4 years ago

By developer's local workflow I mean "my daily/hourly work, running test, modifying code, compiling things, …".

To quote the Tekton mission

Be the industry-standard, cloud-native CI/CD platform components and ecosystem.

My point is only that in that mission statement there is no "local" part, so this is not on tekton to do this, at least as the mission states it for now. Tekton can help but it's on the user and teams using it to do it correctly (but sharing stuff between their task and scripts, …).

Should we change the mission statements ? Maybe, maybe not (I tend to like "simple mission statement"). Should it be the job of some other project to do that (support tekton constructs for local execution) ? Maybe, maybe not (as of my "I tend to" above, I would lean towards another project).

popcor255 commented 4 years ago

I think it depends on what your goals are for development. If you are developing something like a 12 factor app then production should be close as possible to development. So, in this case I agree with @bobcatfish. However, not everything I develop needs to be as polished. :sweat: I think the mission statement might need to be revised depending on how inclusive we are trying to be with tekton.

jlpettersson commented 4 years ago

I like the current Tekton mission - it has a narrow scope. E.g. it should be a good CI/CD-system (executed after git-push) - correct, good UX, easy to get secure, easy to maintain (e.g. upgrade and evolve)

But the developer workflow is indeed a real problem - that we as "business organizations and tekton users" face often - but this problem includes a bit more:

For "one repo - one built artifact" - a Tekton Pipeline consist of a graph of tools and also includes another repo for environment-configuration/infrastructure aka config-repo.
But for a "monorepo - many built artifacts" - a Tekton Pipeline with fewer Tasks - as e.g. Bazel is a dynamic pipeline (in a way) that does multiple commands. Also the config-repo might be included in the monorepo?

The config in config-repos listed above are likely handled by tools like Kustomize and Kpt - but may also be needed by Scaffold when deploying multiple apps to a namespace for confirming correct integration with other apps.

Also, a developer workflow includes more - here it is important for very fast feedback cycles - e.g. incremental builds and incremental deployments - for speed - e.g. using Scaffold, Jib, Telepresence and/or possibly Bazel. But a developer would also like to confirm that the sources will pass the Pipeline before git-push - it is annoying when the local tools doesn't catch the problems that the Pipeline catch.

Ideally as a developer writes code in the IDE - all checks that will be done in a Pipeline - might be triggered asynchronously in the background and if any problems occur - shows the problems in the IDE - but we are not there yet!? - even for e.g. integration-tests. Super fast feedback :)