Introduce support for bundles and solve issues.
For efficiency, the pipeline modifies the DB in different stage types, with multiple stage instances per type.
Since the SMT update is a synchronized step, where the atomic unit is a line (or chain) seen from the CSV files, the synchronization step and serialization needs to happen on a stage type that handles either lines or chains.
We pick the chainToCerts stage type for this.
Steps:
[x] Support certificate bundles again.
[x] Add methods in DB handling or Manager that account for inserters finishing.
[x] Evaluate the current amount of inserted certificates in chainToCerts
[x] If a new bundle is to be created, stop the pipeline, wait until termination, then run OnBundleFinished
[x] Resume the pipeline if it was stopped.
(Done via the WithStallStages option in the pipeline)
Additionally and while reproducing bugs related to bundles and concurrent insertions, some deadlocks have been observed at articuno. Steps:
[ ] Reproduce deadlocks in DB at articuno when the bundle size is small.
Introduce support for bundles and solve issues. For efficiency, the pipeline modifies the DB in different stage types, with multiple stage instances per type. Since the SMT update is a synchronized step, where the atomic unit is a line (or chain) seen from the CSV files, the synchronization step and serialization needs to happen on a stage type that handles either lines or chains. We pick the
chainToCertsstage type for this. Steps:Managerthat account for inserters finishing.chainToCertsOnBundleFinished(Done via the
WithStallStagesoption in the pipeline)Additionally and while reproducing bugs related to bundles and concurrent insertions, some deadlocks have been observed at
articuno. Steps:Deadlocks are not reproducible anymore.
This change is