`ConcurrentModificationException` due to parallel passes

[konradweiss]

The iteration in the ArrayList function next, triggers a ConcurrentModificationException. This apparently happens in: https://github.com/Fraunhofer-AISEC/cpg/blob/f19388e118d1c6ad3711222ef0a55e535bd45378/cpg-core/src/main/kotlin/de/fraunhofer/aisec/cpg/graph/edge/PropertyEdge.kt#L201 where in the map function the iteration over the elements happens. In parallel to the EvaluationOrderGraphPass runs the JavaExternalTypeHierarchyResolver where a modification to the list may happen.

Stacktrace

java.util.concurrent.ExecutionException: java.util.ConcurrentModificationException
    at java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:396)
    at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2073)
    at de.fraunhofer.aisec.cpg_vis_neo4j.Application.call(Application.kt:591)
    at de.fraunhofer.aisec.cpg_vis_neo4j.Application.call(Application.kt:87)
    at picocli.CommandLine.executeUserObject(CommandLine.java:2041)
    at picocli.CommandLine.access$1500(CommandLine.java:148)
    at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2461)
    at picocli.CommandLine$RunLast.handle(CommandLine.java:2453)
    at picocli.CommandLine$RunLast.handle(CommandLine.java:2415)
    at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2273)
    at picocli.CommandLine$RunLast.execute(CommandLine.java:2417)
    at picocli.CommandLine.execute(CommandLine.java:2170)
    at de.fraunhofer.aisec.cpg_vis_neo4j.ApplicationKt.main(Application.kt:632)
Caused by: java.util.ConcurrentModificationException
    at java.base/java.util.ArrayList$Itr.checkForComodification(ArrayList.java:1013)
    at java.base/java.util.ArrayList$Itr.next(ArrayList.java:967)
    at de.fraunhofer.aisec.cpg.graph.edge.PropertyEdge$Companion.unwrap(PropertyEdge.kt:411)
    at de.fraunhofer.aisec.cpg.graph.edge.PropertyEdgeDelegate.getValue(PropertyEdge.kt:376)
    at de.fraunhofer.aisec.cpg.graph.Node.getPrevEOG(Node.kt:159)
    at de.fraunhofer.aisec.cpg.helpers.SubgraphWalker.getEOGPathEdges(SubgraphWalker.kt:232)
    at de.fraunhofer.aisec.cpg.graph.scopes.LoopScope.getStarts(LoopScope.kt:59)
    at de.fraunhofer.aisec.cpg.passes.EvaluationOrderGraphPass.connectCurrentToLoopStart(EvaluationOrderGraphPass.kt:814)
    at de.fraunhofer.aisec.cpg.passes.EvaluationOrderGraphPass.handleDoStatement(EvaluationOrderGraphPass.kt:869)
    at de.fraunhofer.aisec.cpg.passes.EvaluationOrderGraphPass$22.invoke(EvaluationOrderGraphPass.kt:126)
    at de.fraunhofer.aisec.cpg.passes.EvaluationOrderGraphPass$22.invoke(EvaluationOrderGraphPass.kt:126)
    at de.fraunhofer.aisec.cpg.passes.EvaluationOrderGraphPass.createEOG(EvaluationOrderGraphPass.kt:388)
    at de.fraunhofer.aisec.cpg.passes.EvaluationOrderGraphPass.handleSwitchStatement(EvaluationOrderGraphPass.kt:968)
    at de.fraunhofer.aisec.cpg.passes.EvaluationOrderGraphPass$30.invoke(EvaluationOrderGraphPass.kt:134)
    at de.fraunhofer.aisec.cpg.passes.EvaluationOrderGraphPass$30.invoke(EvaluationOrderGraphPass.kt:134)
    at de.fraunhofer.aisec.cpg.passes.EvaluationOrderGraphPass.createEOG(EvaluationOrderGraphPass.kt:388)
    at de.fraunhofer.aisec.cpg.passes.EvaluationOrderGraphPass.handleBlock(EvaluationOrderGraphPass.kt:535)
    at de.fraunhofer.aisec.cpg.passes.EvaluationOrderGraphPass$18.invoke(EvaluationOrderGraphPass.kt:122)
    at de.fraunhofer.aisec.cpg.passes.EvaluationOrderGraphPass$18.invoke(EvaluationOrderGraphPass.kt:122)
    at de.fraunhofer.aisec.cpg.passes.EvaluationOrderGraphPass.createEOG(EvaluationOrderGraphPass.kt:388)
    at de.fraunhofer.aisec.cpg.passes.EvaluationOrderGraphPass.handleFunctionDeclaration(EvaluationOrderGraphPass.kt:323)
    at de.fraunhofer.aisec.cpg.passes.EvaluationOrderGraphPass$5.invoke(EvaluationOrderGraphPass.kt:102)
    at de.fraunhofer.aisec.cpg.passes.EvaluationOrderGraphPass$5.invoke(EvaluationOrderGraphPass.kt:101)
    at de.fraunhofer.aisec.cpg.passes.EvaluationOrderGraphPass.createEOG(EvaluationOrderGraphPass.kt:388)
    at de.fraunhofer.aisec.cpg.passes.EvaluationOrderGraphPass.handleTranslationUnitDeclaration(EvaluationOrderGraphPass.kt:215)
    at de.fraunhofer.aisec.cpg.passes.EvaluationOrderGraphPass$2.invoke(EvaluationOrderGraphPass.kt:95)
    at de.fraunhofer.aisec.cpg.passes.EvaluationOrderGraphPass$2.invoke(EvaluationOrderGraphPass.kt:94)
    at de.fraunhofer.aisec.cpg.passes.EvaluationOrderGraphPass.createEOG(EvaluationOrderGraphPass.kt:388)
    at de.fraunhofer.aisec.cpg.passes.EvaluationOrderGraphPass.accept(EvaluationOrderGraphPass.kt:172)
    at de.fraunhofer.aisec.cpg.passes.EvaluationOrderGraphPass.accept(EvaluationOrderGraphPass.kt:73)
    at de.fraunhofer.aisec.cpg.passes.PassKt$consumeTargets$futures$1$1.get(Pass.kt:300)
    at de.fraunhofer.aisec.cpg.passes.PassKt$consumeTargets$futures$1$1.get(Pass.kt:241)

Possible Solutions

• get a list copy before iterating
• synchronize access to (all) property edges
• Identify why JavaExternalTypeHierarchyResolver access eog Edges and ensure that passes running in parallel are never accessing the same property edges.

[oxisto]

Do you have a reproducible example for this? I think I also encountered something like this sometimes, but I am not sure if the JavaExternalTypeHierarchyResolver is really the culprit.

[konradweiss]

We solved this for now, I did parse the entire openvpn repository and it did happen in different places and not reliably. I will investigate it further.