Provide build time external resource

[squakez]

When we build a project, we have faced certain corner cases where a build time external resource is required (see #4200). We've recently worked on the possibility to alter the maven build by changing the settings (#4568), so we may try to use the same approach and develop a feature where the user can include any resource to be considered at build time.

For example:

-t builder.resource=configmap:my-file.txt

would include the resource in the classpath for any build time need.

[squakez]

fyi @robertofabrizi

@gansheer, wdyt? you have worked in the user profile settings. Do you think we can leverage that development to include any generic resource at build time?

[robertofabrizi]

This would bridge a current gap between what is supported by camel natively, but not by camelk (which is also a bit not that clear for the novices to the technology). Sadly every camel component to handle soap services requires a build time resource, so at the moment none of them is compatible with camelk in native mode.

[squakez]

This would bridge a current gap between what is supported by camel natively, but not by camelk (which is also a bit not that clear for the novices to the technology). At the moment, so many camel native components (mostly related to legacy tech like soap services) are unavailable to use in camelk.

I'd be cautious with such a statement. Only components which requires local "build-time" resources may not work as expected in native mode.

[robertofabrizi]

Umm, I didn't mean to sound harsh or anything, I'll edit my comment to better focus on one specific class of components, but at the moment every single camel component that handles soap services requires build time resources, and therefore can't be used natively with camelk.

[gansheer]

@squakez What you are proposing looks like a lot like what exists in Kamel CLI run command:

--build-property stringArray     Add a build time property or properties file from a path, a config map or a secret (syntax: [my-key=my-value|file:/path/to/my-conf.properties|[configmap|secret]:name]])

Is your idea to replace it by a builder trait ? If so should we keep the 'file:' options?

[squakez]

Not exactly. The ''build-property'' flag transform any plain property or read from configmap/secret/files and then convert into builder trait properties. From build perspective, it always receive a list of text properties which are passed to the maven build accordingly.

This issue would be more similar to the builder.maven-profile where the operator is in charge to read the configmap/secret and append to the pom.xml. What I expect if we implement the builder.resource is to read the configmap in the same way, but create a file on the project classpath. In this way, we can control the build passing the location of the given file which will be available at build time.

[claudio4j]

Is there other use cases besides the xslt component ? For the xslt component, there are two modes to make the xslt file available

• native mode: file available in the jar and classpath
• jvm mode: can be file and http

[gansheer]

After some discussions with @claudio4j and some experiments with the xslt components I think we can this feature can be done.

What I did was the following:

• 1/ read the file content from the configmap
• 2/ create a file in the appropriate folder /tmp/kit-xxxx/src/main/resources
• 3/ ensure it is added in the result of the build (what is injected in the resulting kit container image) and usable

Why this process:

• For 1/ : since we are talking about build time external resource we can't mount the configmap on the operator, that would be difficult to do and would result on unwanted respins of operator/builder pod. Reading content from a secret/configmap without volume mount is already a thing we do for user provided settings.xml, so there is nothing new for us. The user should be mindful of the quantity of configmaps added, but there is low risk as the data stored in a ConfigMap cannot exceed 1 MiB.
• For 2/: the advantage of using /tmp/kit-xxxx/src/main/resource are:
  • this folder should be in the workdir of the operator/builder pod so we have writing permissions and it will be deleted at the end of the build
  • this content of the folder will be added automatically in the camel-k-integration-.jar, therefor available after build
• For 3/: that would need more tests but from what my experiments the files in the camel-k-integration-.jar are available with .to("xslt:example.xslt") This would allow to add files as resources, but build property would still use the --build-property flag as it needs to be added to application.properties.

I feel what we really need is to start with defining use cases/example that could be used to understand if it works of this proposal or if it needs some adaptation.

[claudio4j]

Looks good Gaëlle ! Is the configmap resource to be deleted after the build ? The use case scenarios are important to be able to test the different types of files, for example files with different encodings and not plain text files. One relevant aspect is the context are to be copied two times, so the task should ensure the final file is the same as the original.

[gansheer]

Is the configmap resource to be deleted after the build ?

As we are not in charge of its creation it is like for other external configmap/secret used we do not take care of its lifecycle. We would still need it if we need to re-create the integration kit for any reason. The management of the configmap's lifecycle by the operator is an old issue : https://github.com/apache/camel-k/issues/1235. If a user want to ensure a configmap is not modified it could be declared as immutable but it doesn't avoid any change if it is deleted then recreated.

The use case scenarios are important to be able to test the different types of files, for example files with different encodings and not plain text files. One relevant aspect is the context are to be copied two times, so the task should ensure the final file is the same as the original.

It depends on the code's ability to can manage configmap's data and binaryData. I don't know if binaryData works. And to check if two copies of a same context has the same result there is always digests compute.

[github-actions[bot]]

This issue has been automatically marked as stale due to 90 days of inactivity. It will be closed if no further activity occurs within 15 days. If you think that’s incorrect or the issue should never stale, please simply write any comment. Thanks for your contributions!