Geo fragmentation and pagination don't work properly at the same time - with one-memeber.jsonld dataset

[xdxxxdx]

Describe the bug:

When set:

• view 0 is the pagination
• view 1 is geo fragment

User one-member.json dataset. = >The geo fragment doesn't work

The context and output files: One-member Geo-page.zip

Pre-requisite VSDS-LDESWorkbench-NiFi Docker image tag: 20230214T123440 VSDS-LDES-E2E-testing

• LDES server simulator Docker image tag: 20230214t1503
• LDES client sink Docker image tag: 20230214T1503

VSDS-LDESServer4J Docker image tag: 20230214T1234 MongoDB Docker image tag: 6.0.4

Use the parameters set in: One-memeber.Geo-page.zip

• .env file set to: user.env
• use docker-compose.yml

To Reproduce Follow the test scenario described in : https://github.com/Informatievlaanderen/VSDS-LDES-E2E-testing/tree/main/e2e-test/use-cases/gipod/5.paginate-ldes

1. Run docker compose up -d => 5 docker containers running
2. Login Nifi and put and start the Nifi flow: https://github.com/Informatievlaanderen/VSDS-LDES-E2E-testing/blob/main/e2e-test/use-cases/gipod/5.paginate-ldes/nifi-workflow.json
3. Post the one member data and create alias pointing to the one member data stream. (Modify your create-alias.json accordingly)

   curl -X POST http://localhost:9011/ldes -H 'Content-Type: application/ld+json' -d '@data/one-member.jsonld'
   curl -X POST http://localhost:9011/alias -H "Content-Type: application/json" -d '@create-alias.json'

Expected behavior It should be 4 Geo fragment and 1 pagination created + the root fragment in the mongo db ldesfragment collection

Current behavior Please see: ldesfragment.json in the One-member Geo-page.zip file. There is no Geo fragment generated.

Please see the LdesSever log: Ldes-Serverlog.txt in the One-member Geo-page.zip file

PS I also tried with six-members.jsonld dataset. The output is the pagination doesn't work well, if you set several views. If you set time-based + pagination, there is no issue. if you set pagination + Geo (Geo alone, Geo+time-based), the issue is there

Please let me know if you can reproduce And feel free to correct me if I am wrong

Thanks XD

[Tomvbe]

The first time I tried this locally I ran into the following exception:

2023-02-17T12:26:23.010+01:00 ERROR 60874 --- [nio-8080-exec-9] o.a.c.c.C.[.[.[/].[dispatcherServlet]    : Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed: java.util.ConcurrentModificationException: java.util.ConcurrentModificationException] with root cause

java.util.ConcurrentModificationException: null
    at java.base/java.util.HashMap.computeIfAbsent(HashMap.java:1229) ~[na:na]
    at be.vlaanderen.informatievlaanderen.ldes.server.domain.ldesfragment.services.FragmentationExecutorImpl.retrieveRootFragmentOfView(FragmentationExecutorImpl.java:48) ~[classes/:na]
    at be.vlaanderen.informatievlaanderen.ldes.server.domain.ldesfragment.services.FragmentationExecutorImpl.lambda$executeFragmentation$0(FragmentationExecutorImpl.java:37) ~[classes/:na]
    at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183) ~[na:na]
    at java.base/java.util.HashMap$EntrySpliterator.forEachRemaining(HashMap.java:1858) ~[na:na]
    at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509) ~[na:na]
    at java.base/java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:290) ~[na:na]
    at java.base/java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:754) ~[na:na]
    at java.base/java.util.concurrent.ForkJoinTask.doExec$$$capture(ForkJoinTask.java:387) ~[na:na]
    at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java) ~[na:na]
    at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1311) ~[na:na]
    at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1841) ~[na:na]
    at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1806) ~[na:na]
    at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:177) ~[na:na]

I could find a similar exception in @xdxxxdx her server.log

2023-02-17 10:03:08 2023-02-17T09:03:08.196Z ERROR 1 --- [nio-8080-exec-1] o.a.c.c.C.[.[.[/].[dispatcherServlet]    : Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed: java.util.ConcurrentModificationException: java.util.ConcurrentModificationException] with root cause
2023-02-17 10:03:08 
2023-02-17 10:03:08 java.util.ConcurrentModificationException: null
2023-02-17 10:03:08 at java.base/java.util.HashMap.computeIfAbsent(HashMap.java:1221) ~[na:na]
2023-02-17 10:03:08 at be.vlaanderen.informatievlaanderen.ldes.server.domain.ldesfragment.services.FragmentationExecutorImpl.retrieveRootFragmentOfView(FragmentationExecutorImpl.java:48) ~[ldes-fragmentisers-geospatial-jar-with-dependencies.jar:na]
2023-02-17 10:03:08 at be.vlaanderen.informatievlaanderen.ldes.server.domain.ldesfragment.services.FragmentationExecutorImpl.lambda$executeFragmentation$0(FragmentationExecutorImpl.java:37) ~[ldes-fragmentisers-geospatial-jar-with-dependencies.jar:na]
2023-02-17 10:03:08 at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183) ~[na:na]
2023-02-17 10:03:08 at java.base/java.util.HashMap$EntrySpliterator.forEachRemaining(HashMap.java:1850) ~[na:na]
2023-02-17 10:03:08 at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509) ~[na:na]
2023-02-17 10:03:08 at java.base/java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:290) ~[na:na]
2023-02-17 10:03:08 at java.base/java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:754) ~[na:na]
2023-02-17 10:03:08 at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) ~[na:na]
2023-02-17 10:03:08 at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) ~[na:na]
2023-02-17 10:03:08 at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) ~[na:na]
2023-02-17 10:03:08 at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) ~[na:na]
2023-02-17 10:03:08 at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) ~[na:na]

This is likely not a duplicate of the other geospatial fragmentation bug. When this exception occurs, I am missing data. When I tried again, it succeeded without running in this concurrent exception and all data was present.

[xdxxxdx]

Hello @Tomvbe , Normally the case cannot be close before I validated it. Thanks

[Tomvbe]

reopened to be validated by QA

[xdxxxdx]

Validated on the LDES server docker image: 20230227T0812