Add new endpoints from CFE-18

[MoonBow-1]

ID <30> Reporter <@MoonBow-1>

CFE-18 has been updated, and the endpoint which have been changed and now there is more of them.

Endpoints to do:

• [x] storage/flow
• [x] storage/capture
• [x] host/relp
• [x] host/meta/ip
• [x] host/meta/interface
• [x] host/group
• [x] capture/relp - Only 1 hub works, rest are missing Omrelp
• [x] capture/groups/linkage
• [x] capture/group

[MoonBow-1]

Found out that HostMeta field hostName isn't populated while investigating CFE-18 issue, will be fixed for endpoint "host/meta"

[MoonBow-1]

storage endpoints are WIP, starting to test them

[MoonBow-1]

Storage endpoints have been created, data verified to be correctly entered to CFE-18 by @Nefarious46

[MoonBow-1]

Refactored parser to support multiple types of captures, introduced and fixed unwanted iteration with that. "Exclusion" parameters changed as well to choose correct parser for capture in commit a056b7d7

[MoonBow-1]

RELP captures do not have Omrelp, so flow is not accessible that way, but endpoint requests that. @Nefarious46 Could you comment on this?

[Nefarious46]

RELP captures do not have Omrelp, so flow is not accessible that way, but endpoint requests that. @Nefarious46 Could you comment on this?

Possible under the hood configuration? Default configuration of sorts?

[kortemik]

relp captures should have it but they have not been set there. there is a separate configuration system for relp so it is not using the json.

perhaps they should be set into the config?

On Thu, Oct 12, 2023 at 1:03 PM Nefarious46 @.***> wrote:

RELP captures do not have Omrelp, so flow is not accessible that way, but endpoint requests that. @Nefarious46 https://github.com/Nefarious46 Could you comment on this?

Possible under the hood configuration? Default configuration of sorts?

— Reply to this email directly, view it on GitHub https://github.com/teragrep/cfe_31/issues/3#issuecomment-1759311303, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAGX2OQCRU67MUUUIUUYV63X666ARANCNFSM6AAAAAA5PNEN4Q . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[MoonBow-1]

relp captures should have it but they have not been set there. there is a separate configuration system for relp so it is not using the json. perhaps they should be set into the config? … On Thu, Oct 12, 2023 at 1:03 PM Nefarious46 @.> wrote: RELP captures do not have Omrelp, so flow is not accessible that way, but endpoint requests that. @Nefarious46 https://github.com/Nefarious46 Could you comment on this? Possible under the hood configuration? Default configuration of sorts? — Reply to this email directly, view it on GitHub <#3 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAGX2OQCRU67MUUUIUUYV63X666ARANCNFSM6AAAAAA5PNEN4Q . You are receiving this because you are subscribed to this thread.Message ID: @.>

All right, I'll write it to the CFE-31 conf. Could you send me some information via DMs where these values are stored/gotten from at the moment so I can get them?

[MoonBow-1]

One of the hubs has been updated to contain the Omrelp information, and parser now works with those values. Will confirm the data in CFE-18

[MoonBow-1]

Had a wrong URL for the Relp Host endpoint, fixed that and now they are inserted correctly

[MoonBow-1]

RELP based endpoints should be now done, will confirm with @Nefarious46 tomorrow. If green light is given, will move the ticket to QA

[MoonBow-1]

If a capture cannot be added due to missing data, it is still possible for the parser to parse Hosts for that group, which should be fixed to not do that.

[MoonBow-1]

If a capture cannot be added due to missing data, it is still possible for the parser to parse Hosts for that group, which should be fixed to not do that.

This has been fixed in commit d678469b

[MoonBow-1]

All the endpoints are now populated, going to request a review of the changes and results.

NOTE: currently there's data that is not sent due to missing RELP configs on the hubs

[MoonBow-1]

PR created internally with ID <40>.

[MoonBow-1]

@kortemik Could you review current PRs/issue when you have time? Order would be:

1. 5 (check if the package name is now correct)

2. 4

3. This one

[MoonBow-1]

All PRs merged