apache / camel-k

Apache Camel K is a lightweight integration platform, born on Kubernetes, with serverless superpowers
https://camel.apache.org/camel-k
Apache License 2.0
863 stars 345 forks source link

Give the operator the possibility to build an Integration without running #5588

Open squakez opened 4 months ago

squakez commented 4 months ago

This issue is collecting some possible idea around the feature discussed in this chat [1]. The requirement has been popping in several occasions, so I think it deserve some space to be officially discussed with any possible pro and cons.

I was thinking that, with this feature, we can fully support the 5th factor [2] of 12 factors cloud development which mandates the need to clearly separate build, release and run.

The architecture we have in place *may be already able to support such feature as it already works in a step by step pipeline. The Integration move from a Build, Deploy and Run phase accordingly. The only thing is that we don't have a proper way to tell the operator to just Build and put in a kind of Ready (to run) Integration phase.

I think we can enable this feature by letting the operator just stop once the container backing the Integration has been successfully built and pushed to the container registry (ie, with some Integration configuration, even an annotation). Then, the same operator (or any other using the same registry) could be able to run the Integration just executing the Deployment configuration (ie, those traits in charge to alter Kubernetes configuration).

I don't see any immediate cons (unless there is a big development effort or important changes that we discover in a deeper analysis). The work on this feature may also cross with #5539, above all if we decide to move the source directly into the built container.

[1] https://camel.zulipchat.com/#narrow/stream/257299-camel-k/topic/Using.20Camel.20K.20operator.20to.20build.20an.20image [2] https://12factor.net/build-release-run

*to be verified with a deep analysis.

gansheer commented 4 months ago

This feature proposal sound very good. If you don't mind I have a couple of questions.

I think we can enable this feature by letting the operator just stop once the container backing the Integration has been successfully built and pushed to the container registry (ie, with some Integration configuration, even an annotation). Then, the same operator (or any other using the same registry) could be able to run the Integration just executing the Deployment configuration (ie, those traits in charge to alter Kubernetes configuration).

When your are saying "integration" what are you talking about ? Is it the Integration CRD ? Also how do you see the second phase being activated in terms of design ?

squakez commented 4 months ago

When your are saying "integration" what are you talking about ? Is it the Integration CRD ?

Yes, it's the CR.

Also how do you see the second phase being activated in terms of design ?

In theory this should be a drift the operator would capture by normally watching at the Integration. Ie, when the user decide to run, then, the operator would move the phase from the hypotetical "Ready" status to "Deploy" status and continuing with the rest of the process smoothly (at least, while talking on paper). Mind that the "normal" execution can continue as is, ie, making the operator to move from "BuildKit" phase to "Deploy", unless the user has configured the Integration to perform such pit-stop.

mdebarros commented 3 months ago

I am very glad to see this proposal.

In fact I would go a step further and suggest that the concerns of "build" & "run" should be completely separated - i.e. Operator has two distinct deployable services/images that can each run independently.

This would enable:

  1. More Flexibility - Only deploying the "build" Operator will allow "DevOps" clusters to be used to build Sourcefull Integrations, which can be utilised in deployment-type environments running the "run" Operator.
  2. Hardened Security - This will allow the running of "immutable" images for both Sourcefull and Sourceless deployments in high-security environments (i.e. Production).

I just want to reiterate that the two capabilities should be two distinct separate services, one managing the "build" and the other managing "deployments".

Alternatively, If a single operator could be configured to run in some combination of "build" or "run" only, it would raise security concerns since either capability could still be enabled in a high-security environment (specifically the "build" capability being a concern here) - and thus the seperation of concerns here would be mostly nullified.

Regardless, the difficulty here would how to "hand-over" the deployment - which could be solvable by the "run" operator waiting for Integrations to reach a specific state - i.e. Integration has been successfully built and is ready for deployment. There could also be a "trait" or a property that would also indicate if this "Integration" should be handled by the "run" operator once the desired state has been reached.

github-actions[bot] commented 2 days ago

This issue has been automatically marked as stale due to 90 days of inactivity. It will be closed if no further activity occurs within 15 days. If you think that’s incorrect or the issue should never stale, please simply write any comment. Thanks for your contributions!