NETOBSERV-1500 : Refactoring of transform network API

OlivierCazade commented 8 months ago

Description

Created different type for the different transform rules

Previously the NetworkTransformRule had all fields at the same level, but they were not all used by all transform network types and the Parameter field was used differently depending on the type.

The struct go from this:

type NetworkTransformRule struct {
    Input           string        `yaml:"input,omitempty" json:"input,omitempty" doc:"entry input field"`
    Output          string        `yaml:"output,omitempty" json:"output,omitempty" doc:"entry output field"`
    Type            string        `yaml:"type,omitempty" json:"type,omitempty" enum:"TransformNetworkOperationEnum" doc:"one of the following:"`
    Parameters      string        `yaml:"parameters,omitempty" json:"parameters,omitempty" doc:"parameters specific to type"`
    Assignee        string        `yaml:"assignee,omitempty" json:"assignee,omitempty" doc:"value needs to assign to output field"`
    KubernetesInfra *K8sInfraRule `yaml:"kubernetes_infra,omitempty" json:"kubernetes_infra,omitempty" doc:"Kubernetes infra rule specific configuration"`
    Kubernetes      *K8sRule      `yaml:"kubernetes,omitempty" json:"kubernetes,omitempty" doc:"Kubernetes rule specific configuration"`
}

To this :


type NetworkTransformRule struct {
    Type            string                 `yaml:"type,omitempty" json:"type,omitempty" enum:"TransformNetworkOperationEnum" doc:"one of the following:"`
    KubernetesInfra *K8sInfraRule          `yaml:"kubernetes_infra,omitempty" json:"kubernetes_infra,omitempty" doc:"Kubernetes infra rule configuration"`
    Kubernetes      *K8sRule               `yaml:"kubernetes,omitempty" json:"kubernetes,omitempty" doc:"Kubernetes rule configuration"`
    AddSubnet       *NetworkAddSubnetRule  `yaml:"add_subnet,omitempty" json:"add_subnet,omitempty" doc:"Add subnet rule configuration"`
    AddLocation     *NetworkGenericRule    `yaml:"add_location,omitempty" json:"add_location,omitempty" doc:"Add location rule configuration"`
    AddIPCategory   *NetworkGenericRule    `yaml:"add_ip_category,omitempty" json:"add_ip_category,omitempty" doc:"Add ip category rule configuration"`
    AddService      *NetworkAddServiceRule `yaml:"add_service,omitempty" json:"add_service,omitempty" doc:"Add service rule configuration"`
}

Now every rule type has its own structure.

Dependencies

n/a

Checklist

If you are not familiar with our processes or don't know what to answer in the list below, let us know in a comment: the maintainers will take care of that.

[x] Will this change affect NetObserv / Network Observability operator? If not, you can ignore the rest of this checklist.
[x] Is this PR backed with a JIRA ticket? If so, make sure it is written as a title prefix (in general, PRs affecting the NetObserv/Network Observability product should be backed with a JIRA ticket - especially if they bring user facing changes).
[ ] Does this PR require product documentation?
- [ ] If so, make sure the JIRA epic is labelled with "documentation" and provides a description relevant for doc writers, such as use cases or scenarios. Any required step to activate or configure the feature should be documented there, such as new CRD knobs.
[ ] Does this PR require a product release notes entry?
- [ ] If so, fill in "Release Note Text" in the JIRA.
[ ] Is there anything else the QE team should know before testing? E.g: configuration changes, environment setup, etc.
- [ ] If so, make sure it is described in the JIRA ticket.
QE requirements (check 1 from the list):
- [ ] Standard QE validation, with pre-merge tests unless stated otherwise.
- [x] Regression tests only (e.g. refactoring with no user-facing change).
- [ ] No QE (e.g. trivial change with high reviewer's confidence, or per agreement with the QE team).

OlivierCazade commented 8 months ago

/hold This is breaking change in the configuration API, we need to synchronize

codecov[bot] commented 8 months ago

Codecov Report

Attention: Patch coverage is 65.21739% with 24 lines in your changes are missing coverage. Please review.

Project coverage is 67.22%. Comparing base (ab65bc4) to head (d688fe3). Report is 3 commits behind head on main.

Files	Patch %	Lines
pkg/pipeline/transform/transform_network.go	45.71%	15 Missing and 4 partials :warning:
pkg/pipeline/transform/kubernetes/enrich.go	77.27%	2 Missing and 3 partials :warning:

Additional details and impacted files

```diff @@ Coverage Diff @@ ## main #580 +/- ## ========================================== + Coverage 66.97% 67.22% +0.24% ========================================== Files 104 104 Lines 7446 7459 +13 ========================================== + Hits 4987 5014 +27 + Misses 2162 2143 -19 - Partials 297 302 +5 ``` | [Flag](https://app.codecov.io/gh/netobserv/flowlogs-pipeline/pull/580/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=netobserv) | Coverage Δ | | |---|---|---| | [unittests](https://app.codecov.io/gh/netobserv/flowlogs-pipeline/pull/580/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=netobserv) | `67.22% <65.21%> (+0.24%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=netobserv#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

jotak commented 7 months ago

/lgtm

jotak commented 7 months ago

@OlivierCazade can you edit the description to briefly explain what are the breaking changes, and give an example of how to convert an old config to a new one? This would help users who are hit by the breaking change to quickly figure out how to handle it

jotak commented 7 months ago

I'm adding no-qe / no-doc as there's no user facing changes - I tested myself along with https://github.com/netobserv/network-observability-operator/pull/562

memodi commented 6 months ago

/ok-to-test

github-actions[bot] commented 6 months ago

New image: quay.io/netobserv/flowlogs-pipeline:380f2f0

It will expire after two weeks.

To deploy this build, run from the operator repo, assuming the operator is running:

USER=netobserv VERSION=380f2f0 make set-flp-image

jotak commented 6 months ago

/lgtm

OlivierCazade commented 6 months ago

/approve

openshift-ci[bot] commented 6 months ago

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: OlivierCazade

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files: - ~~[OWNERS](https://github.com/netobserv/flowlogs-pipeline/blob/main/OWNERS)~~ [OlivierCazade] Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment

netobserv / flowlogs-pipeline