LDES Client version materialization: don't ask for versionOf path

[sandervd]

Is your feature request related to a problem? Please describe. The LDES already communicates the versionOf path that should be used, we shouldn't ask for it as config.

Describe the solution you'd like Remove the config, and check the metadata

[pj-cegeka]

The same can apply to timestamp path used to determine the member order within a fragment

[jobulcke]

As a data consumer, I don't want to configure properties that are present in the ES

Description

Currently, there are a few components that requires config properties that are actually already present in the ES that is about to be consumed, therefore, additional logic needs to be added to this work.

In the documentation is also stated that the multiple urls provided to the client must be part of the same LDES, and this additional work happens on the exact place where this could be checked, this check should be added here.

Discussed solution:

These ES props will be stored in memory, as this only needs to be fetched on intialization of the MemberSupplier, which start the whole initialization process.

When fetching the first fragment, either a ViewSpecification (VSpec => ES or View definition), or a TreeNodeSpecification (TSpec, => Fragment), are returned.

A VSpec should contain the following triples and therefore no further action needs to be taken:

<collection-uri> a ldes:EventStream
    ldes:timestampPath <timestamp-path>
    ldes:versionOfPath <versionOf-path>

A TSpec should contain the following triples, and this information must be used for the next step in the properties fetching process:

<fragment-identifier> <dcterms:isPartOf> <collection-uri>

With the collectionUri, a new request must be sent out that will be used for retrieving the ES props.

The flow can be represented as the following:

flowchart TD
;
    Urls["Urls[0]"] --> StartingNode[StartingTreeNode];
    StartingNode --> DecisionDiamond{VSpec or TSpec};
    DecisionDiamond -->|VSpec| Extract[Extract ES properties \n out of response]
    DecisionDiamond -->|TSpec| Request[Make new request \n using dcterms:isPartOf]
    Request --> Extract
    Extract --> Pass[Pass properties through interested parts of the Client]

Acceptance Criteria

• The config properties timestamp-path and materialization.versionOf-path must be ignored
• The e2e tests must be updated, so the omission can be verified
• The properties still can be used after the application is restarted
• The process is only executed ONCE after startup, for the first provided url, as the multiple urls must be part of the same LDES
• The documentation for LDIO must be updated
• The PropertyDescriptors in NiFi must be deleted