netenglabs / suzieq

Using network observability to operate and design healthier networks
https://www.stardustsystems.net/
Apache License 2.0
780 stars 104 forks source link

Poller fix: write deletions with the same timestamp #881

Closed LucaNicosia closed 1 year ago

LucaNicosia commented 1 year ago

Test inclusion requirements

In case the PR contains an enhancement or a new platform/service support, some tests have to be added for the new functionality:

For additional information about tests, follow this link

Description

If the same uncaolesced file has more than one timestamp in the same file we might have issues when coalesced and uncaolesced data coexists in the data-directory.

In order to avoid the problem above, we must force the timestamp of all records written to be the same. Since all the records to write will have the same timestamp, we are going to set that timestamp also for the deletions. If instead no data was returned from the device (f.e. the device was unreachable) we use the current timestamp for the deleted records.

Type of change

New Behavior

All the deletions in the same uncaolesced file will have the same timestamp

Contrast to Current Behavior

A new timestamp for each deleted record will be present in the same uncoalesced file

Double Check