Fix issue with inconsistent GenericMap output

[jotak]

Description

The FLP GenericMap provided by the agent differs in types depending on the exporter being used, especially with DirectFLP each records size is smaller than when used with protobuf, due to protobuf using uint32 almost everywhere. This is not a problem in most cases as FLP generally converts that in JSON, but it breaks the FLP IPFIX export which expects strict types.

This commit brings more consistency between the FLP-direct mode and the protobuf mode, by sharing the same GenericMap encoding function. Which means the protobuf decode is now done in two steps: first proto-to-flow.Record, then flow.Record-to-GenericMap.

Dependencies

FLP update of the IPFIX exporter: https://github.com/netobserv/flowlogs-pipeline/pull/634

Checklist

If you are not familiar with our processes or don't know what to answer in the list below, let us know in a comment: the maintainers will take care of that.

• [ ] Will this change affect NetObserv / Network Observability operator? If not, you can ignore the rest of this checklist.
• [ ] Is this PR backed with a JIRA ticket? If so, make sure it is written as a title prefix (in general, PRs affecting the NetObserv/Network Observability product should be backed with a JIRA ticket - especially if they bring user facing changes).
• [ ] Does this PR require product documentation?
  • [ ] If so, make sure the JIRA epic is labelled with "documentation" and provides a description relevant for doc writers, such as use cases or scenarios. Any required step to activate or configure the feature should be documented there, such as new CRD knobs.
• [ ] Does this PR require a product release notes entry?
  • [ ] If so, fill in "Release Note Text" in the JIRA.
• [ ] Is there anything else the QE team should know before testing? E.g: configuration changes, environment setup, etc.
  • [ ] If so, make sure it is described in the JIRA ticket.
• QE requirements (check 1 from the list):
  • [ ] Standard QE validation, with pre-merge tests unless stated otherwise.
  • [ ] Regression tests only (e.g. refactoring with no user-facing change).
  • [ ] No QE (e.g. trivial change with high reviewer's confidence, or per agreement with the QE team).

[github-actions[bot]]

New image: quay.io/netobserv/netobserv-ebpf-agent:e1397a0

It will expire after two weeks.

To deploy this build, run from the operator repo, assuming the operator is running:

USER=netobserv VERSION=e1397a0 make set-agent-image

[codecov[bot]]

Codecov Report

Attention: Patch coverage is 95.38462% with 3 lines in your changes are missing coverage. Please review.

Project coverage is 33.17%. Comparing base (58d01d9) to head (dffe40a). Report is 1 commits behind head on main.

Files	Patch %	Lines
pkg/decode/decode_protobuf.go	94.82%	2 Missing and 1 partial :warning:

Additional details and impacted files

```diff @@ Coverage Diff @@ ## main #297 +/- ## ========================================== - Coverage 36.74% 33.17% -3.57% ========================================== Files 42 40 -2 Lines 3813 3587 -226 ========================================== - Hits 1401 1190 -211 + Misses 2334 2322 -12 + Partials 78 75 -3 ``` | [Flag](https://app.codecov.io/gh/netobserv/netobserv-ebpf-agent/pull/297/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=netobserv) | Coverage Δ | | |---|---|---| | [unittests](https://app.codecov.io/gh/netobserv/netobserv-ebpf-agent/pull/297/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=netobserv) | `33.17% <95.38%> (-3.57%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=netobserv#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[msherif1234]

any cpu impact for the additional processing with ur scale test ?

[msherif1234]

why this is marked as breaking change ?

[jotak]

why this is marked as breaking change ?

That's a breaking change because the types in the GenericMap output change, as you can see here: https://github.com/netobserv/netobserv-ebpf-agent/pull/297/files#diff-5b98b198dc9b743f2c1c2d470d443d9ec2ed38b4ee15777cb857584ddacce657 For instance "Proto" was u8 in the inner model, cast as uint32 in protobuf, previously with the grpc exporter it was still uint32 in the GenericMap but now it's back to uint8 in that Map.

So, consumers who would cast explicitly to uint32 would now have to cast to uint8

[msherif1234]

/lgtm

[jotak]

any cpu impact for the additional processing with ur scale test ?

Slight increase of CPU in one of the tests (0.167 -> 0.172) and memory as well (671MB -> 679 MB) (i picked the one run that seems more relevant as it showed a similar number of flows per seconds). I'd say it's in reasonable bounds...

[jotak]

/approve marking no-qe as there's no visible output

[openshift-ci[bot]]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jotak

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files: - ~~[OWNERS](https://github.com/netobserv/netobserv-ebpf-agent/blob/main/OWNERS)~~ [jotak] Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment

[jotak]

/retest-required

[jotak]

/retest