ess-dmsc / kafka-to-nexus

(Mirror: moved to https://gitlab.esss.lu.se/ecdc/ess-dmsc/kafka-to-nexus) Writes NeXus files from experiment data streamed through Apache Kafka.
BSD 2-Clause "Simplified" License
2 stars 4 forks source link

Ecdc 3447 filewriter attribute module #703

Closed ggoneiESS closed 11 months ago

ggoneiESS commented 1 year ago

Issue

ECDC-3447

Description of work

Created a new module with four-byte identifier 'mdat' that currently processes start_time and stop_time and is expandable for future processing and other messages sent directly to Kafka.

The start time and stop time that already exist within the filewriter in other locations are written specifically at the correct times (in JobCreator.cpp where the start time is 'obvious' and StreamController.cpp, which is the method always processed regardless of filewriter state, except in the most catastrophic crashes).

Relevant tests have been added.

Checklist

cow-bot commented 1 year ago

Code Coverage (Lines Exec Cover) TOTAL 5727 3853 67% For more detail see Cobertura report in Jenkins interface

cow-bot commented 1 year ago

Code Coverage (Lines Exec Cover) TOTAL 5727 3847 67% For more detail see Cobertura report in Jenkins interface

cow-bot commented 1 year ago

Code Coverage (Lines Exec Cover) TOTAL 5731 3853 67% For more detail see Cobertura report in Jenkins interface

cow-bot commented 1 year ago

Code Coverage (Lines Exec Cover) TOTAL 5731 3851 67% For more detail see Cobertura report in Jenkins interface

cow-bot commented 1 year ago

Code Coverage (Lines Exec Cover) TOTAL 5732 3857 67% For more detail see Cobertura report in Jenkins interface

cow-bot commented 1 year ago

Code Coverage (Lines Exec Cover) TOTAL 5732 3854 67% For more detail see Cobertura report in Jenkins interface

cow-bot commented 1 year ago

Code Coverage (Lines Exec Cover) TOTAL 5732 3851 67% For more detail see Cobertura report in Jenkins interface

cow-bot commented 1 year ago

Code Coverage (Lines Exec Cover) TOTAL 5732 3853 67% For more detail see Cobertura report in Jenkins interface

cow-bot commented 1 year ago

Code Coverage (Lines Exec Cover) TOTAL 5733 3853 67% For more detail see Cobertura report in Jenkins interface

cow-bot commented 1 year ago

Code Coverage (Lines Exec Cover) TOTAL 5732 3851 67% For more detail see Cobertura report in Jenkins interface

cow-bot commented 1 year ago

Code Coverage (Lines Exec Cover) TOTAL 5732 3853 67% For more detail see Cobertura report in Jenkins interface

cow-bot commented 1 year ago

Code Coverage (Lines Exec Cover) TOTAL 5732 3853 67% For more detail see Cobertura report in Jenkins interface

ggoneiESS commented 1 year ago

I have no idea why this passed this time when I tried to rebase to an earlier commit, VScode complained, and I merged manually...dd337d7 and ec7eff8 should otherwise be identical!

EDIT: Timeouts changed to 60s. Not sure where delay coming from, locally there were no issues so hard to determine.

cow-bot commented 1 year ago

Code Coverage (Lines Exec Cover) TOTAL 5732 3853 67% For more detail see Cobertura report in Jenkins interface

cow-bot commented 1 year ago

Code Coverage (Lines Exec Cover) TOTAL 5730 3852 67% For more detail see Cobertura report in Jenkins interface

cow-bot commented 1 year ago

Code Coverage (Lines Exec Cover) TOTAL 5730 3854 67% For more detail see Cobertura report in Jenkins interface

cow-bot commented 1 year ago

Code Coverage (Lines Exec Cover) TOTAL 5730 3852 67% For more detail see Cobertura report in Jenkins interface

cow-bot commented 1 year ago

Code Coverage (Lines Exec Cover) TOTAL 5734 3863 67% For more detail see Cobertura report in Jenkins interface

cow-bot commented 1 year ago

Code Coverage (Lines Exec Cover) TOTAL 5734 3861 67% For more detail see Cobertura report in Jenkins interface

cow-bot commented 1 year ago

Code Coverage (Lines Exec Cover) TOTAL 5734 3864 67% For more detail see Cobertura report in Jenkins interface

cow-bot commented 1 year ago

Code Coverage (Lines Exec Cover) TOTAL 5734 3861 67% For more detail see Cobertura report in Jenkins interface

cow-bot commented 1 year ago

Code Coverage (Lines Exec Cover) TOTAL 5734 3858 67% For more detail see Cobertura report in Jenkins interface

cow-bot commented 1 year ago

Code Coverage (Lines Exec Cover) TOTAL 5734 3857 67% For more detail see Cobertura report in Jenkins interface

cow-bot commented 12 months ago

Code Coverage (Lines Exec Cover) TOTAL 5734 3863 67% For more detail see Cobertura report in Jenkins interface

cow-bot commented 12 months ago

Code Coverage (Lines Exec Cover) TOTAL 5734 3856 67% For more detail see Cobertura report in Jenkins interface

cow-bot commented 11 months ago

Code Coverage (Lines Exec Cover) TOTAL 5710 3839 67% For more detail see Cobertura report in Jenkins interface

cow-bot commented 11 months ago

Code Coverage (Lines Exec Cover) TOTAL 5710 3840 67% For more detail see Cobertura report in Jenkins interface

ggoneiESS commented 11 months ago

This has the discussed changes in, with a new templated method writemetadata added to the mdat module. This means the flatbuffer creation is removed, and instead the relevant info is passed directly to the method. To achieve the call to the method, a static_cast is deployed but iff it is the mdat module so there is no need to worry about catching going into an undefined state.

cow-bot commented 11 months ago

Code Coverage (Lines Exec Cover) TOTAL 5702 3822 67% For more detail see Cobertura report in Jenkins interface

cow-bot commented 11 months ago

Code Coverage (Lines Exec Cover) TOTAL 5703 3840 67% For more detail see Cobertura report in Jenkins interface