The Draco Generic Enabler is an alternative data persistence mechanism for managing the history of context. It is based on Apache NiFi and is a dataflow system based on the concepts of flow-based programming. It supports powerful and scalable directed graphs of data routing, transformation, and system mediation logic and also offers an intuitive graphical interface
Note: submitting the PR in draft status since there are still some things to finish (see TODOs below), but new features are already in good shape and tested on a real deployment.
Proposed changes
Globally extend the NGSI-LD support of the NGSIToPostgreSQL processor and also do some global upgrades (NiFi version, Java version):
chore: upgrade NiFi version to 1.14.0 (only tested for NGSIToPostgreSQL processor)
chore: set base Java version to 11 for nifi-ngsi-bundle
chore: replace part of System.out statements by log statements
feat: extract keys to ignore in payloads into a constant list for improved maintainability
feat: add support for multi-attributes (section 4.5.5 of the NGSI-LD spec) to avoid too long (and truncated) names
a configurable prefix can be removed from the dataset ids
feat: add support for sub-attributes of all types of attributes (was previously only supported for properties)
feat: use types as much as possible for columns (e.g. TIMESTAMPTZ for dates or NUMERIC for numeric values)
currently using TEXT (default), TIMESTAMPTZ and NUMERIC (more could be added in the future)
it won't break existing installations because columns already exist, but they could benefit from such a new feature
TODOs before removing the draft status
align NGSIv2 code where needed (e.g., in getValuesForInsert, listOfFields)
add some specific tests for the new features
upgrade bundle version (to which version?)
Types of changes
What types of changes does your code introduce to the project: Put an x in the boxes that apply
[ ] Bugfix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
Note: submitting the PR in draft status since there are still some things to finish (see TODOs below), but new features are already in good shape and tested on a real deployment.
Proposed changes
Globally extend the NGSI-LD support of the NGSIToPostgreSQL processor and also do some global upgrades (NiFi version, Java version):
TODOs before removing the draft status
getValuesForInsert
,listOfFields
)Types of changes
What types of changes does your code introduce to the project: Put an
x
in the boxes that applyChecklist