SolidLabResearch / LDES-in-SOLID-Semantic-Observations-Replay

1 stars 2 forks source link

The location of the inbox of the LDES Stream is not patched upon creation of a new LDP Container to store the replayed events. #32

Open argahsuknesib opened 6 months ago

argahsuknesib commented 6 months ago

Hello @svrstich , I have been testing the recent patch made by you for the real-time-replayer.

When replaying the observations one by one, the events get added to the latest inbox of the LDES in LDP, which works fine. However, after creation of the new container, the timestamps still use the original timestamp instead of the realtime ones. when triggering the /advancePushPointerToTheEnd.

Screenshot from 2024-03-15 10-41-25

My config file looks like this,

{
    "port": "3001",
    "loglevel": "debug",
    "logname": "WEB API",
    "__comment__" : "enter the location of your dataset folder below this in datasetFolders field. e.g. /home/data/",
    "datasetFolders": "/home/kush/Code/feather-RDF-mapper/data/rdfData",
    "credentialsFileName": null,
    "lilURL": "http://localhost:3000/aggregation_pod/realTimeReplay/",
    "treePath": "https://saref.etsi.org/core/hasTimestamp",
    "chunkSize": 10,
    "bucketSize": 50,
    "targetResourceSize": 1024,
    "batchSize": 50,
    "original_timestamp": false
}
argahsuknesib commented 6 months ago

I checked the code, and it is possible that the /advanceAndPushObservationPointerToTheEnd still uses the original timestamp implementation with the naive algorithm.

argahsuknesib commented 6 months ago

Hmm, but if I just ignore the /advancePushPointerToTheEnd and just use /startAutoReplay this works fine. I think we can specify in README the steps to do both real-time and original-timestamp replay.

However, even when using the /startAutoReplay the latest inbox location is not patched when a new container is created.

image

svrstich commented 5 months ago

Hi Kush, I believe this is intended behaviour. /advancePushPointerToTheEnd tries to submit all observations as fast as possible into the pod. If we were to adapt the timestamp, either all observation would be more or less at the same time, or we would be sending observations with a timestamp in the future towards the pod.

argahsuknesib commented 5 months ago

Hello @svrstich , Yes I agree with /advancesPushPointerToTheEnd's intention. However, the inbox is still not updated for which I am working on in my fork.