[RFE] Dynamic File-Based Input for Rules

[brunoborges]

Enable Konveyor / Kantra to accept files as input to rules, allowing these files to be parsed and utilized as dynamic input parameters. This feature would enhance flexibility by supporting the inclusion of various data formats (e.g., JSON, YAML, CSV) for rule execution. Users could upload a configuration or data file, which the system would parse to extract necessary parameters, thus allowing more complex and customized rule conditions without manual input. This capability would streamline workflows, especially in scenarios requiring detailed configuration or large data sets.

[konveyor-ci-bot[bot]]

This issue is currently awaiting triage. If contributors determine this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance. The triage/accepted label can be added by org members.

[shawn-hurley]

Hello,

I think that we would only accept xml (run through the shim) and yaml files. today we can accept xml rules if you use the `--rules`` flag.

[shawn-hurley]

actually just tested and you can pass in a yaml file as well:

 analyze --rules ../rulesets/default/generated/cloud-readiness/70-local-storage.windup.yaml --input ../testing-apps/tackle-testapp-public --maven-settings ../settings.xml -o output --overwrite

[shawn-hurley]

Let me know if that will cover the usecase, I am not fully clear on how a CSV would be used and I don't know if we want to go about supporting a different type of rule format.

[brunoborges]

Well, not really. Let me give you an example of a use case:

Customer has a list of domains/URLs they want to check if they are hardcoded in the source code.

Today, one option is to write a custom rule. Alternatively, we would like to see the ability of the Rule parser to read data from external files in addition to the rule file definition.

Let's look at the localhost rule. Right now, it looks for localhost, but what if we wanted to look for other domains, such as mycompany or staging or prodenv.

when:
    builtin.filecontent:
      filePattern: .*\.(java|properties|jsp|jspf|tag|xml|txt|yaml)
      pattern: http(s)?://((localhost)|(127\.0\.0\.1))+(:[0-9]+)?(/.*)?

In the code above, I'd like to provide a parameter instead of localhost so that it would expand with a list of domains coming from an external file.

Another common scenario is imports.

[shawn-hurley]

Thank you, this feature request is making much more sense to me now!

It sounds like you are asking for some way loop over an external source of some interpolated pattern in the rules, creating new rules from that source for each pattern that is found in the external source.

I'll let @rromannissen weigh in on the priority, but I think I understand the ask now and think that it is worth exploring if the priority is high enough

[rromannissen]

I'm a bit concerned about how this request might affect the concept of rules as "automated pieces of knowledge", as this would leave the door open to some uncertainty about how a rule might behave. By design, rules are expected to be very explicit about the antipatterns to look for and provide a solution for those antipatterns, and if this was implemented, rules could stop being self contained and behave differently depending on certain input at runtime, which could potentially be a bit non deterministic. I totally see how this could be convenient for some custom rules development scenarios, but it would be kind of dangerous for rules that come out of the box with Konveyor, especially for potential downstream distributions, as it would be hard to support those rules if their behavior is not preestablished.

Nevertheless, I would definitely not oppose to this being implemented, but considering that the same results could be achieved at the moment via custom rules by adding additional when criteria entries, I don't think our team would be tackling this on the short to mid term, although we would be open to contributions on this direction.