teragrep / cfe_31

0 stars 0 forks source link

CaptureMeta duplicates return another nameif the data is already in the DB when sending #20

Closed MoonBow-1 closed 7 months ago

MoonBow-1 commented 8 months ago

This was related to an old ticket for CFE-18, which has now been fixed, and it breaks the send method for CaptureMeta.

If the name has been marked as a duplicate, the response from the server will be the original name of the duplicate if the data is the same. This also applies to CaptureMetas with a different name, but the same data

MoonBow-1 commented 8 months ago

Found out that this bug was introduced in commit 843c990a, going to investigate the reason

MoonBow-1 commented 8 months ago

Problem is the location of the CaptureMeta send method inside the GenerateFiles. If the Meta is sent right after constructing it in the generateFiles, it will result in duplicate names in CFE-18. If this is done in a for-each loop in the send method of the GenerateFile, this will result in much less errors from CFE-18.

Doesn't work properly:

            final CaptureMeta captureMeta = captureDefinitionEntry.getCaptureMeta(this);
            captureMeta.generateNewNameForDuplicateCaptureMeta(captureMetaSet);

            captureMetaSet.add(captureMeta);
            captureMeta.send();
MoonBow-1 commented 8 months ago

The breaking changes were rolled back up to commit 85e50f70. This will require more refactoring to get it working somehow.

Maybe the CaptureMetaMap is still a viable option, and in the sendAll method, get the required CaptureMeta for the CaptureDefinition by the processingType field, send the captureMeta --> construct a new CaptureDefinition with the new processingType response from CFE-18.

MoonBow-1 commented 8 months ago

Maybe the CaptureMetaMap is still a viable option, and in the sendAll method, get the required CaptureMeta for the CaptureDefinition by the processingType field, send the captureMeta --> construct a new CaptureDefinition with the new processingType response from CFE-18.

This has been implemented/fixed in commit 6fb22645.

There are some captureMeta errors still showing up, but these are pretty rare --> investigating

MoonBow-1 commented 8 months ago

Looked at this issue today, moved the CaptureMetaMap to be passed from main, so it persists between different dirs. CFE-18 still has some issues, so will be continuing on this tomorrow.

MoonBow-1 commented 8 months ago

There we some changes on CFE-18, where it doesn't accept CaptureMetas with the same name.

Problem: CaptureMetas are sent every time before Captures, resulting in errors. Solution: implement a Set of already sent CaptureMetas to not send duplicates.

Implemented in commit 863ae879

MoonBow-1 commented 8 months ago

Implemented a fix on commit 2535b514, where the CaptureMetaMap's generateNewName method bypassed its own put and putIfAbsent methods, and used the Map's own those, resulting in the name field being populated in the Map's values.

captureMetaMap.put(captureMeta.getName(), new CaptureMeta());

fixed to:

put(captureMeta);

Old version: 352 CaptureMetas, of which ~250 sent New version: 151 captureMetas, of which 116 sent