[feature request]Ability to export application behavior as policies for runtime.

[HoogWater]

Is your feature request related to a problem? Please describe. This feature request is an extention not currently existing

Describe the solution you'd like I would like to be able to export a policy ruleset based on a successful test in order to use that ruleset as input for anomaly detection and runtime enforcement in the production environment. For example as Falco ruleset or OPA policies.

Describe alternatives you've considered I have not considered any alternatives. I think it is a awesome capability make this tool even more usable.

Additional context Because a defined testset is probably either a preferred/allowed or forcedfail/denied flow of your application, the output of any test can be used as a application dependency map or behavior map, this can be used in your production environment as security/anomaly rules of allowed flows or anomalous behavior, therefore being able to block any activity that has not been observed in your test environment. Creating a more secure deployment defined already from your build/test flow(shift-left)

[kdhamric]

This would be pretty awesome. I have used a test tool before (Applitools) which was used to do image comparisons to pass or fail tests. Took a baseline screenshot at a particular point in a selenium test - the first time it is the 'baseline', future runs were compared to it. When there was significant deviation, the test would fail. The user was shown the baseline and 'new version' and was allowed to accept the new version as the baseline if desired. Was curious how similar to what you are envisioning this is? Also - I believe you discussed this with @adnanrahic yesterday at Kubecon - if you are on the floor this morning, stop by again - would love to discuss it with you in person.

[bestrand]

+1 on this! I think the case I have been thinking of is the same, just a different output type so I am adding my thoughts out loud here :) I guess it would involve some scanning so that would be a seperate part.

I have been for some time searching for a tool where one could define rulesets which are checked against traces and send notifications if there are violations. From my searches at least I haven’t found any. TraceQL has arrived now, but I haven't had time to look much into it.

After visiting the tracetest booth at KubeCon I realised that this in a test context translates to the assertions tracetest already does. Thinking of this in the context of policy / security one could look at the Test object as a Policy, and get notified via the assertion outcome of a trace analysis on traces coming in.

The policy (type: Policy) could be:

• Abnormal function calls
• Known vulnerabilities like suddenly SQL injection patterns from one service to another (bypassed protection in an LB) in the call chain.
  • This could be a policy set maintained by the community which one could activate as default
• Auto generated Policy definitions from looking at e.g Controllers in spring and define what an app accepts as “incoming traffic”, or allowed "outgoing db traffic / statements"

I don’t know the technical implications of doing these types of “scans”. Since we already have the tracing data it would be great to be able scan it given the level of detail one could look at. For example these scenarios:

• Function A from package B in my java code is allowed to do this db statement, notify if any other DB statements has occurred. Taking the assertion part here it would be essentially doing notify !(already defined test assertion)

• One could also say which type of traffic is allowed as incoming traffic to a database to maybe detect a frontend suddenly doing requests directly to DB

As I said I don’t know the technical implications here, just thinking out loud.

Pasting a yaml I have written earlier to reflect some ideas on what the scan tool would check (not in this format necessarily) :)

...
  - name: service_a
    allowed_incoming_traffic:
      - type: http
        method: POST
        path: /api/records
        allowed_parameters:
          - id
          - name
    allowed_outgoing_traffic:
      - type: database
        database_name: mydb1
        allowed_functions:
          - com.example.ServiceA.getRecords
        allowed_queries:
          - SELECT * FROM mytable WHERE column1 = :param1
  - name: service_b
    allowed_incoming_traffic:
      - type: http
        method: POST
        path: /api/records/:id
        allowed_parameters:
          - id
          - name
    allowed_outgoing_traffic:
      - type: database
        database_name: mydb1
        allowed_functions:
          - com.example.ServiceB.updateRecord
        allowed_queries:
          - UPDATE mytable SET column1 = :param1 WHERE id = :id
  - name: service_c
    allowed_incoming_traffic:
      - type: http
        method: GET
        path: /api/records
        allowed_query_params:
          - id
          - name
    allowed_outgoing_traffic:
      - type: database
        database_name: mydb2
        allowed_functions:
          - com.example.ServiceC.getRecords
        allowed_queries:
          - SELECT * FROM mytable2 WHERE column1 = :param1

   ...

Time to wrap this one up, and looking forward to more discussions on the subject!

[adnanrahic]

Hi @bestrand! Thanks a lot for adding your insight here. It's definitely something we will look into. Once @kdhamric and the engineering team wake up, we'll get back to you on this ASAP. They're all in the western hemisphere.

[kdhamric]

Thanks @bestrand! The team will be discussing this tomorrow in our planning meeting. I wanted to post a couple related issues:

1469 Standard "trace problem" definition

1470 Define common trace "patterns / issues / problems"

by @agardnerIT

[bestrand]

Thanks alot for the fast feedback, and those issues are great!

I will add thoughts in those issues going forward to keep this issue focused on the feature request @HoogWater has proposed.

For learning Go and how traces work / are stored I have created a project: https://github.com/bestrand/tracedog/

Under the examples folder I have added different yaml files which express my ideas better than sentences I guess. Going forward I will try to learn by implementing functionality which can use those files, and hopefully be able to make small contributions to this tool.

[danielbdias]

Hi @bestrand!

Thanks for your reply! I saw your repo, and there are good ideas in the examples folder! It remembers a little bit about a discussion that we had in the past about rules.

We are discussing internally how to build a PoC of this and as soon as we have an idea about it we will return here. During this time feel free to ping us about these ideas!