ldes members are duplicated

[zargha]

The Problem

The problem occurred while testing the e2e 20220705.demo1. The problem is seen during the Scenario: data set contains 2 fragments. beta.jsonld cannot be replicated to ldes server. Therefore, the NiFi workflow was stopped. First LdesClient has been started and then InvokeHTTP. This results in total of 750 ldes members whereas it should be alfa.jsonld + beta.jsonld = 500 members

Fragments uploaded:

curl -X POST http://localhost:9001/ldes -H 'Content-Type: application/json-ld' -d '@gipod-server-simulator/data/scenario3/alfa.jsonld'
curl -X POST http://localhost:9001/ldes -H 'Content-Type: application/json-ld' -d '@gipod-server-simulator/data/scenario3/beta.jsonld'
{"content-type":"application/json-ld","cache-control":"public, max-age=604800, immutable","id":"/api/v1/ldes/mobility-hindrances?generatedAtTime=2022-05-20T09:58:15.867Z"}{"content-type":"application/json-ld","cache-control":"public, max-age=604800, immutable","id":"/api/v1/ldes/mobility-hindrances?generatedAtTime=2022-05-25T10:22:45.82Z"}%                                                                                                                  

Environment

Components under test

• ghcr.io/informatievlaanderen/ldes-server:20220711T1053
• vsds/ldes-client-workflow
• ghcr.io/informatievlaanderen/ldes-server-simulator:20220629t1521
• mongo

  $ lsb_release -a
  No LSB modules are available.
  Distributor ID: Ubuntu
  Description:    Ubuntu 20.04.4 LTS
  Release:        20

  $ docker --version
  Docker version 20.10.17, build 100c701
  $ docker compose version
  Docker Compose version v2.6.0

  $ git branch
  feature/VSDSPUB-62-e2e-test

  Screenhots:

  gipod-server-simulator:

mongodb:

ldes-server: ldesserverduplicate.txt

Steps to reproduce

1. Setup the demo as described here
2. Scenario: replication has not yet started
3. Scenario: replication has just started
4. Scenario: data set contains 2 fragments
5. Stop NiFi workflow
6. Start LdesClient worflow
7. Start InvokeHTTP workflow

The problem occurs during step 7 where duplicate fragments are seen in mongodb.

[WLefever-Cegeka]

Issue should be fixed in #120

[zargha]

The issue has been tested against the image: ghcr.io/informatievlaanderen/ldes-server:20220713T1625 and no duplication has been seen.