Open droot opened 3 years ago
Imperative function run is now eval: https://kpt.dev/book/04-using-functions/02-imperative-function-execution
Regarding metafunctions, modification of Kptfiles, and generation of pipelines: https://kpt.dev/book/04-using-functions/02-imperative-function-execution?id=mutate-meta-resources
Per-function filters are discussed in #2364. krmignore is discussed in #1733 and #517.
If Kptfile is not passed as part of the ResourceList by default, is it essentially like one of these other types of filters except that it's applied by default and disabled by --include-meta-resources? If --include-meta-resources is applied to kpt fn source, does it also need to be applied to kpt fn eval invocations that read from stdin? Is there a way to specify it in a declarative pipeline?
Also, it's unclear whether ConfigMaps and pseudo-resources used for function inputs for particular functions via configPath can be passed in the ResourceList to other functions: https://kpt.dev/book/04-using-functions/01-declarative-function-execution https://kpt.dev/book/04-using-functions/02-imperative-function-execution?id=mutate-meta-resources
Use cases that have come up include:
At least the first two cases also have needed top-down preprocessing (preorder execution) prior to bottom-up rendering (post-order execution) during declarative execution.
cc @justinsb
Thanks for the detailed comment @bgrant0607 . This is helpful as we are beginning the design for supporting filtering in fn
commands. (#2015 )
Will try to answer part of the comment here:
If Kptfile is not passed as part of the ResourceList by default, is it essentially like one of these other types of filters except that it's applied by default and disabled by --include-meta-resources?
this is fairly accurate in context of fn eval
and fn source
. We still need to rationalize this with fn render
. Quick thought suggests that all the use-cases for processing meta-resources in render
actually fits the use-cases for pre-processing step.
If --include-meta-resources is applied to kpt fn source, does it also need to be applied to kpt fn eval invocations that read from stdin?
I think both answers (yes, no) are possible in this case, but optimizing for the common CUJs should drive the answer. Quick thought suggests --include-meta-resources
doesn't need to be applied to fn eval
invocations reading from stdin
because we can assume resources on stdin
are already filtered (if one was using fn source
or previous fn eval
).
Is there a way to specify it in a declarative pipeline?
I think the need for this has a big overlap with pre-processing step requirements I think.
I need to think some more on the second part.
Kind of related, I'd like kpt fn render to be able to take an entire package with a Kptfile with a pipeline from stdin.
Why? Because I always want to re-run render after other mutations in order to enforce invariants. I want to treat the pipeline like mutating + validating admission control.
Every time we clone a package to create a variant of the package, we need to specialize the variant somehow.
The most basic thing that needs to be done in pretty much all cases is personalization: change the names of things: https://github.com/kubernetes/community/blob/master/contributors/design-proposals/architecture/declarative-application-management.md#configuration-customization-and-variant-generation
We do this for the Kptfile's metadata.name field automatically. I would like to use that as a standard input for a variant constructor pattern. This pattern would use the first mutator in the Kptfile to do the basic variant specialization required by the package. We may also want to think about how to use the name as input to other standard name-changing functions, such as set-namespace, ensure-name-substring, and so on.
In general, for our use cases, I always want --include-meta-resources to be true, in source, eval, render, wherever. We need to be able to view and change the info in the Kptfile. It makes more sense to me to filter config objects in the SDKs or in function-specific filtering parameters.
@phanimarupaka I described my use case above
There is significant change in "kpt fn run" workflows with introduction of pipeline construct. Following tasks need to be accomplished: