Model of Data in Clearing House

International-Data-Spaces-Association / InformationModel

The Information Model of the International Data Spaces implements the IDS reference architecture as an extensible, machine readable and technology independent data model.

Apache License 2.0

64 stars 37 forks source link

Model of Data in Clearing House #532

Open kragall opened 2 years ago

kragall commented 2 years ago

To store data in the Clearing House a Connector creates a LogMessage and sends it together with data as the payload to the Clearing House. The Clearing House stores this LogMessage together with meta-data, such as a timestamp when it stored the data or the id under which data can be retrieved. This is illustrated here:

Log Entry drawio

The LogEntry contains both LogMessage and meta-data.

Currently, when a Connector wants to retrieve data from the Clearing House it can query for it, and it will receive the stored LogMessage. The meta-data that is part of the LogEntry is not returned to the Connector. For various reasons it is desirable for the Connector to access the meta-data.

Would it be possible to add something like LogEntry to the Information Model? Or is there another solution how LogMessage and additional meta-data may be grouped together in a way that a receiving connector can process without additional logic?

JohannesLipp commented 2 years ago

Thank you for raising this! Could you please state the additional fields you need? A solution could indeed add additional fields to LogMessage, or create a complete new class LogEntry, if required. The first option is preferable, if no additional logic should be applied.

kragall commented 2 years ago

The meta-data fields we store currently are:

transaction_id: This is basically a counter with a unique (unique in the database) number for each stored message.
process_id: This is a string that describes the process under which the message is logged. Think of it as the label of the folder under which the message has been filed together with other messages.
document_id: This is a unique (unique in the process) id in the form of a uuid for the stored message
timestamp: This is the timestamp recording the date when this message was logged by the CH
hash: This information is used for integrity protection in the CH
client_id: This is the unique id of the client that sent the stored message taken from the DAT the client sent with the original LogMessage

JohannesLipp commented 2 years ago

Task: 1) Create new class LogEntry that holds the given metadata fields, and 2) Link from LogEntry to the existing LogMessage class via a new property.

PHochmann commented 2 years ago

See https://github.com/International-Data-Spaces-Association/InformationModel/tree/feature/log-entry

PHochmann commented 2 years ago

Updated requirements (as given by @kragall ):

[x] timestamp: This is the timestamp recording the date when this message was logged by the CH
[x] transaction_id: This is basically a counter with a unique (unique in the database) number for each stored message.
[x] process_id: This is a string that describes the process under which the message is logged. Think of it as the label of the folder under which the message has been filed together with other messages.
[x] document_id: This is a unique (unique in the process) id in the form of a uuid for the stored message
[x] client_id: This is the unique id of the client that sent the stored message taken from the DAT the client sent with the original LogMessage
[x] server_id: This is the unique id of the server that stored message in form of the id of private key
[x] hash: This information is used for integrity protection in the CH
[x] signature: This contains a cryptographic signature of the data
[x] payload: This optional field contains the payload of the original LogMessage