search

fair-acc / opendigitizer

Generic Digitizer Framework based on OpenCMW
GNU Lesser General Public License v3.0
6 stars 3 forks source link

EPIC: Refactoring low-level GR 4.0 API #42

Closed ivan-cukic closed 7 months ago

ivan-cukic commented 1 year ago
RalphSteinhagen commented 1 year ago

Elimination of RxCpp as a Core OpenCMW/OpenDigitizer Dependency

The original OpenDigitizer concept involved two types of signal processing:

  1. sample-by-sample-based, predominantly in the low-level front-end/services, and
  2. processing of chunked-data, which was done both in the low-level service and notably also at the UI level.

GNU Radio (GR) has always been a good match for the first type of signal processing because it supports a feature-rich suite of high-performance C++-based signal processing blocks and algorithms that can be both programmatically and graphically defined through a flow-graph-type representation that nicely bridged the gap between high-performance C++ developers and graphical interaction for domain experts with insufficient C++ expertise. However, with GR 3.10 and earlier versions, this type of interaction was limited to sample-by-sample processing.

Therefore, the second type of post-processing related to chunked data (e.g., transient-recording and/or multiplexed signal like) needed to be implemented through a 'DataSet'-based processing/transform akin to those found in modern I&M devices. This has been prototyped in the Java-based Chart-FX and OpenCMW projects. Initial investigations found that the reactive pattern and it's C++ implementation RxCpp was a good match for this purpose, but required a graphical graph-based representation component similar to what already existed for GR. However, RxCpp became increasingly stale and not actively maintained nor modernised.

Meanwhile, FAIR was contacted by the GR leadership, who were open to accepting new ideas and requirements for the upcoming GR 4.0 version. This jump-started a productive conversation and a graph prototype, and eventually, a commitment to integrating these as well as modern C++ concepts as part of the upcoming GR 4.0.

The proposed improvements to GR, notably the new type-strict circular buffer, unlock a slimmer API due to the use of modern C++20 concepts. This, in turn, improved the performance of the buffers by 10 to 100-fold and the basic block processing through the introduction of SIMD-based vectorization and parallelization, compile-time merging of nodes, design/preference of 'composition to inheritance,' and minimization of unnecessary virtual inheritance to the absolute necessary. Notably, the type-safe buffers also enable arbitrary types, including the DataSet concept, to be transmitted via these buffers.

As a result, the need for the flow-graph API for RxCpp and a lot of other related boilerplate code, and compatibility with gr companion for both sample-by-sample and chunked-data-based processing were eliminated and are now covered by the targeted new GNU Radio 4.0 implementation.

This thus closes/modifies the following issues: