updated README and yml files

KalmanMeth commented 6 months ago

Description

The stand-alone ebpf and flp were implemented against an older version of flp. This PR updateds the README and yml files to work with the current flp.

Dependencies

n/a

Checklist

If you are not familiar with our processes or don't know what to answer in the list below, let us know in a comment: the maintainers will take care of that.

[ ] Will this change affect NetObserv / Network Observability operator? If not, you can ignore the rest of this checklist.
[ ] Is this PR backed with a JIRA ticket? If so, make sure it is written as a title prefix (in general, PRs affecting the NetObserv/Network Observability product should be backed with a JIRA ticket - especially if they bring user facing changes).
[ ] Does this PR require product documentation?
- [ ] If so, make sure the JIRA epic is labelled with "documentation" and provides a description relevant for doc writers, such as use cases or scenarios. Any required step to activate or configure the feature should be documented there, such as new CRD knobs.
[ ] Does this PR require a product release notes entry?
- [ ] If so, fill in "Release Note Text" in the JIRA.
[ ] Is there anything else the QE team should know before testing? E.g: configuration changes, environment setup, etc.
- [ ] If so, make sure it is described in the JIRA ticket.
QE requirements (check 1 from the list):
- [ ] Standard QE validation, with pre-merge tests unless stated otherwise.
- [ ] Regression tests only (e.g. refactoring with no user-facing change).
- [ ] No QE (e.g. trivial change with high reviewer's confidence, or per agreement with the QE team).

KalmanMeth commented 6 months ago

@jpinsonneau The flp-daemonset-cap.yml doesn't yet work properly. Some permissions probably need to be adjusted.

codecov[bot] commented 6 months ago

Codecov Report

Attention: 24 lines in your changes are missing coverage. Please review.

Comparison is base (6f1d70a) 33.86% compared to head (a023a01) 34.06%. Report is 1 commits behind head on main.

Files	Patch %	Lines
pkg/exporter/proto.go	0.00%	16 Missing and 2 partials :warning:
pkg/flow/deduper.go	85.00%	2 Missing and 1 partial :warning:
pkg/utils/utils.go	50.00%	2 Missing and 1 partial :warning:

Additional details and impacted files

```diff @@ Coverage Diff @@ ## main #249 +/- ## ========================================== + Coverage 33.86% 34.06% +0.20% ========================================== Files 39 39 Lines 3479 3508 +29 ========================================== + Hits 1178 1195 +17 - Misses 2233 2243 +10 - Partials 68 70 +2 ``` | [Flag](https://app.codecov.io/gh/netobserv/netobserv-ebpf-agent/pull/249/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=netobserv) | Coverage Δ | | |---|---|---| | [unittests](https://app.codecov.io/gh/netobserv/netobserv-ebpf-agent/pull/249/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=netobserv) | `34.06% <54.71%> (+0.20%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=netobserv#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

jpinsonneau commented 6 months ago

I'm getting flows in FLP after applying the new changes:

Starting flowlogs-pipeline:
=====
Build Version: -dirty
Build Date: 2024-01-17 17:34

Using configuration:
{
    "PipeLine": "[{\"name\":\"ingest\"},{\"follows\":\"ingest\",\"name\":\"enrich\"},{\"follows\":\"enrich\",\"name\":\"loki\"}]",
    "Parameters": "[{\"ingest\":{\"grpc\":{\"port\":9999},\"type\":\"grpc\"},\"name\":\"ingest\"},{\"name\":\"enrich\",\"transform\":{\"network\":{\"rules\":[{\"input\":\"SrcAddr\",\"output\":\"SrcK8S\",\"type\":\"add_kubernetes\"},{\"input\":\"DstAddr\",\"output\":\"DstK8S\",\"type\":\"add_kubernetes\"}]},\"type\":\"network\"}},{\"name\":\"loki\",\"write\":{\"loki\":{\"labels\":[\"SrcK8S_Namespace\",\"SrcK8S_OwnerName\",\"DstK8S_Namespace\",\"DstK8S_OwnerName\",\"FlowDirection\"],\"staticlabels\":{\"app\":\"netobserv-flowcollector\"},\"timestamplabel\":\"TimeFlowEndMs\",\"timestampscale\":\"1ms\",\"url\":\"http://loki.netobserv.svc:3100\"},\"type\":\"loki\"}}]",
    "MetricsSettings": "",
    "Health": {
        "Address": "0.0.0.0",
        "Port": "8080"
    },
    "Profile": {
        "Port": 0
    }
}
time=2024-01-19T12:41:23Z level=debug msg=opts.PipeLine = [{"name":"ingest"},{"follows":"ingest","name":"enrich"},{"follows":"enrich","name":"loki"}] 
time=2024-01-19T12:41:23Z level=debug msg=stages = [{ingest } {enrich ingest} {loki enrich}] 
time=2024-01-19T12:41:23Z level=debug msg=params = [{ingest 0xc000130f00 <nil> <nil> <nil> <nil>} {enrich <nil> 0xc0002bd080 <nil> <nil> <nil>} {loki <nil> <nil> <nil> <nil> 0xc0002bd0e0}] 
time=2024-01-19T12:41:23Z level=info msg=using default metrics settings
time=2024-01-19T12:41:23Z level=debug msg=entering SetupElegantExit
time=2024-01-19T12:41:23Z level=debug msg=registered exit signal channel
time=2024-01-19T12:41:23Z level=debug msg=exiting SetupElegantExit
time=2024-01-19T12:41:23Z level=info msg=StartServerAsync: addr = :9090 component=prometheus
time=2024-01-19T12:41:23Z level=debug msg=entering newPipelineFromIngester
time=2024-01-19T12:41:23Z level=debug msg=stages = [{ingest } {enrich ingest} {loki enrich}] 
time=2024-01-19T12:41:23Z level=debug msg=configParams = [{ingest 0xc000130f00 <nil> <nil> <nil> <nil>} {enrich <nil> 0xc0002bd080 <nil> <nil> <nil>} {loki <nil> <nil> <nil> <nil> 0xc0002bd0e0}] 
time=2024-01-19T12:41:23Z level=debug msg=stage = ingest
time=2024-01-19T12:41:23Z level=debug msg=findStageType: stage = ingest
time=2024-01-19T12:41:23Z level=debug msg=pipeline = [0xc00024c7e0]
time=2024-01-19T12:41:23Z level=debug msg=stage = enrich
time=2024-01-19T12:41:23Z level=debug msg=findStageType: stage = enrich
time=2024-01-19T12:41:23Z level=debug msg=starting kubernetes informers, waiting for synchronization
time=2024-01-19T12:41:23Z level=debug msg=kubernetes informers started
time=2024-01-19T12:41:23Z level=debug msg=starting kubernetes metadata informers, waiting for synchronization
time=2024-01-19T12:41:23Z level=debug msg=kubernetes metadata informers started
time=2024-01-19T12:41:23Z level=debug msg=pipeline = [0xc00024c7e0 0xc0001a2a10]
time=2024-01-19T12:41:23Z level=debug msg=stage = loki
time=2024-01-19T12:41:23Z level=debug msg=findStageType: stage = loki
time=2024-01-19T12:41:23Z level=debug msg=entering NewWriteLoki component=write.Loki
time=2024-01-19T12:41:23Z level=debug msg=pipeline = [0xc00024c7e0 0xc0001a2a10 0xc0001a2d90]
time=2024-01-19T12:41:23Z level=debug msg=pipeline = [0xc00024c7e0 0xc0001a2a10 0xc0001a2d90]
time=2024-01-19T12:41:23Z level=info msg=connecting stages: ingest --> enrich
time=2024-01-19T12:41:23Z level=info msg=connecting stages: enrich --> loki
time=2024-01-19T12:41:27Z level=debug msg=Ingested 875 records component=ingest.GRPCProtobuf
time=2024-01-19T12:41:27Z level=debug msg=Ingested 920 records component=ingest.GRPCProtobuf
time=2024-01-19T12:41:32Z level=debug msg=Ingested 1069 records component=ingest.GRPCProtobuf
time=2024-01-19T12:41:32Z level=debug msg=Ingested 1027 records component=ingest.GRPCProtobuf
time=2024-01-19T12:41:37Z level=debug msg=Ingested 1125 records component=ingest.GRPCProtobuf
time=2024-01-19T12:41:37Z level=debug msg=Ingested 844 records component=ingest.GRPCProtobuf
time=2024-01-19T12:41:42Z level=debug msg=Ingested 1094 records component=ingest.GRPCProtobuf

KalmanMeth commented 6 months ago

I'm getting flows in FLP after applying the new changes:

To get rid of these messages, we need to change log-level from debug to info or error.

KalmanMeth commented 6 months ago

Fixed perms in a023a01 and changed the instructions order to create namespace properly at first command.

@KalmanMeth note that if you are using oc command, ommiting labels section for pod-security is actually working !

Using kubectl, you must specify labels 🙃

@jpinsonneau Can you merge your commit with this PR?

jpinsonneau commented 6 months ago

@jpinsonneau Can you merge your commit with this PR?

It's already part of this PR, we can merge as is to me. Thanks @KalmanMeth !

openshift-ci[bot] commented 6 months ago

[APPROVALNOTIFIER] This PR is APPROVED

Approval requirements bypassed by manually added approval.

This pull-request has been approved by:

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files: - **[OWNERS](https://github.com/netobserv/netobserv-ebpf-agent/blob/main/OWNERS)** Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment

netobserv / netobserv-ebpf-agent