search

yuzutech / kroki

Creates diagrams from textual descriptions!
https://kroki.io
MIT License
2.78k stars 206 forks source link

Ditaa is slow and can block the event loop thread #557

Open ggrossetie opened 3 years ago

ggrossetie commented 3 years ago

Ditaa is called directly from the gateway server and as a result can block the event loop thread:

WARNING: Thread Thread[vert.x-eventloop-thread-0,5,main]=Thread[vert.x-eventloop-thread-0,5,main] has been blocked for 77376 ms, time limit is 2000 ms
io.vertx.core.VertxException: Thread blocked
    at java.base@11.0.6/java.util.HashMap$TreeNode.balanceInsertion(Unknown Source)
    at java.base@11.0.6/java.util.HashMap$TreeNode.putTreeVal(Unknown Source)
    at java.base@11.0.6/java.util.HashMap.putVal(Unknown Source)
    at java.base@11.0.6/java.util.HashMap.put(Unknown Source)
    at java.base@11.0.6/java.util.HashSet.add(Unknown Source)
    at app//org.stathissideris.ascii2image.text.CellSet.add(CellSet.java:63)
    at app//org.stathissideris.ascii2image.text.TextGrid.seedFillOld(TextGrid.java:1196)
    at app//org.stathissideris.ascii2image.text.TextGrid.fillContinuousArea(TextGrid.java:1177)
    at app//org.stathissideris.ascii2image.graphics.Diagram.<init>(Diagram.java:165)
    at app//org.stathissideris.ascii2image.core.CommandLineConverter.convertToImage(CommandLineConverter.java:144)
    at app//org.stathissideris.ascii2image.core.CommandLineConverter.doConvert(CommandLineConverter.java:110)
    at app//org.stathissideris.ascii2image.core.CommandLineConverter.convert(CommandLineConverter.java:97)
    at app//io.kroki.server.service.Ditaa.convert(Ditaa.java:60)
    at app//io.kroki.server.service.Ditaa.convert(Ditaa.java:51)
    at app//io.kroki.server.service.DiagramHandler.convert(DiagramHandler.java:152)

At the very least, we should use a worker thread (or a worker verticle) to execute the task.

As far as I know it's not possible to reliably cancel/terminate a thread. For instance, if the thread is running uninterruptedly (for instance in an infinite loop) using Thread.interrupt() (or even the unsafe Thread.stop()) won't work.

Another way, would be to execute Ditaa as an external program (i.e., executing Ditaa as a Java program from a Java program using Runtime.exec()). That would probably increase latency as we will need to start a new JVM each time. GraalVM could help reduce the boot/start time if we manage to produce a native image.

Please note that the same thing can happen with PlantUML.

Feel free to join this conversation and submit ideas :hugs:

ggrossetie commented 3 years ago

With https://github.com/yuzutech/kroki/pull/645 Ditaa should be significantly faster (still a bit slow but we can't do much more here...)

ggrossetie commented 3 years ago

We still have issues when ditaa is called from the PlantUML endpoint (because PlantUML is not using the optimized version by @pepijnve).

io.vertx.core.VertxException: Thread blocked
    at app//org.stathissideris.ascii2image.text.TextGrid.getTestingSubGrid(TextGrid.java:159)
    at app//org.stathissideris.ascii2image.text.TextGrid.matchesAny(TextGrid.java:1187)
    at app//org.stathissideris.ascii2image.text.TextGrid.isIntersection(TextGrid.java:1243)
    at app//org.stathissideris.ascii2image.text.TextGrid.isPointCell(TextGrid.java:789)
    at app//org.stathissideris.ascii2image.graphics.CompositeDiagramShape.growEdgesFromCell(CompositeDiagramShape.java:127)
    at app//org.stathissideris.ascii2image.graphics.CompositeDiagramShape.createOpenFromBoundaryCells(CompositeDiagramShape.java:85)
    at app//org.stathissideris.ascii2image.graphics.Diagram.<init>(Diagram.java:352)
    at java.base@11.0.9.1/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at java.base@11.0.9.1/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
    at java.base@11.0.9.1/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
    at java.base@11.0.9.1/java.lang.reflect.Constructor.newInstance(Unknown Source)
    at app//net.sourceforge.plantuml.ditaa.PSystemDitaa.exportDiagramNow(PSystemDitaa.java:127)
    at app//net.sourceforge.plantuml.AbstractPSystem.exportDiagram(AbstractPSystem.java:146)
    at app//io.kroki.server.service.Plantuml.convert(Plantuml.java:249)
    at app//io.kroki.server.service.Plantuml.lambda$convert$2(Plantuml.java:211)
    at app//io.kroki.server.service.Plantuml$$Lambda$182/0x00000001002a1040.handle(Unknown Source)
    at app//io.vertx.core.impl.ContextImpl.lambda$null$0(ContextImpl.java:179)
    at app//io.vertx.core.impl.ContextImpl$$Lambda$185/0x00000001002a2040.handle(Unknown Source)
    at app//io.vertx.core.impl.AbstractContext.dispatch(AbstractContext.java:96)
    at app//io.vertx.core.impl.ContextImpl.lambda$executeBlocking$1(ContextImpl.java:177)
    at app//io.vertx.core.impl.ContextImpl$$Lambda$184/0x00000001002a1c40.run(Unknown Source)
    at app//io.vertx.core.impl.TaskQueue.run(TaskQueue.java:76)
    at app//io.vertx.core.impl.TaskQueue$$Lambda$103/0x00000001001adc40.run(Unknown Source)
    at java.base@11.0.9.1/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.base@11.0.9.1/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at app//io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
    at java.base@11.0.9.1/java.lang.Thread.run(Unknown Source)
io.vertx.core.VertxException: Thread blocked
    at java.base@11.0.9.1/java.util.regex.Pattern.matcher(Unknown Source)
    at app//org.stathissideris.ascii2image.text.GridPattern.isMatchedBy(GridPattern.java:114)
    at app//org.stathissideris.ascii2image.text.GridPatternGroup.isAnyMatchedBy(GridPatternGroup.java:44)
    at app//org.stathissideris.ascii2image.text.TextGrid.matchesAny(TextGrid.java:686)
    at app//org.stathissideris.ascii2image.text.TextGrid.matchesAny(TextGrid.java:1188)
    at app//org.stathissideris.ascii2image.text.TextGrid.isIntersection(TextGrid.java:1243)
    at app//org.stathissideris.ascii2image.text.TextGrid.followCell(TextGrid.java:1112)
    at app//org.stathissideris.ascii2image.graphics.CompositeDiagramShape.growEdgesFromCell(CompositeDiagramShape.java:139)
    at app//org.stathissideris.ascii2image.graphics.CompositeDiagramShape.createOpenFromBoundaryCells(CompositeDiagramShape.java:85)
    at app//org.stathissideris.ascii2image.graphics.Diagram.<init>(Diagram.java:352)
    at java.base@11.0.9.1/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at java.base@11.0.9.1/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
    at java.base@11.0.9.1/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
    at java.base@11.0.9.1/java.lang.reflect.Constructor.newInstance(Unknown Source)
    at app//net.sourceforge.plantuml.ditaa.PSystemDitaa.exportDiagramNow(PSystemDitaa.java:127)
    at app//net.sourceforge.plantuml.AbstractPSystem.exportDiagram(AbstractPSystem.java:146)
    at app//io.kroki.server.service.Plantuml.convert(Plantuml.java:249)
    at app//io.kroki.server.service.Plantuml.lambda$convert$2(Plantuml.java:211)
    at app//io.kroki.server.service.Plantuml$$Lambda$182/0x00000001002a1040.handle(Unknown Source)
    at app//io.vertx.core.impl.ContextImpl.lambda$null$0(ContextImpl.java:179)
    at app//io.vertx.core.impl.ContextImpl$$Lambda$185/0x00000001002a2040.handle(Unknown Source)
    at app//io.vertx.core.impl.AbstractContext.dispatch(AbstractContext.java:96)
    at app//io.vertx.core.impl.ContextImpl.lambda$executeBlocking$1(ContextImpl.java:177)
    at app//io.vertx.core.impl.ContextImpl$$Lambda$184/0x00000001002a1c40.run(Unknown Source)
    at app//io.vertx.core.impl.TaskQueue.run(TaskQueue.java:76)
    at app//io.vertx.core.impl.TaskQueue$$Lambda$103/0x00000001001adc40.run(Unknown Source)
    at java.base@11.0.9.1/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.base@11.0.9.1/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at app//io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
    at java.base@11.0.9.1/java.lang.Thread.run(Unknown Source)
ggrossetie commented 3 years ago

I think it will be safer (but unfortunately a bit slower) to execute ditaa as a CLI from Java. That way we won't have to worry about infinite loops and unoptimized code because the process will be killed after 5 seconds.

If we can produce a native binary of ditaa with GraalVM that would be good since we want fast startup (on a short lived process)

ggrossetie commented 1 year ago

Another thread blocked:

io.vertx.core.VertxException: Thread blocked
  at java.base@11.0.15/java.util.concurrent.ConcurrentHashMap.get(Unknown Source)
  at java.desktop@11.0.15/sun.font.FontDesignMetrics.getMetrics(Unknown Source)
  at java.desktop@11.0.15/sun.java2d.SunGraphics2D.getFontMetrics(Unknown Source)
  at app//org.stathissideris.ditaa.graphics.FontMeasurer.getAscent(FontMeasurer.java:80)
  at app//org.stathissideris.ditaa.graphics.FontMeasurer$2.test(FontMeasurer.java:110)
  at app//org.stathissideris.ditaa.graphics.FontMeasurer.deriveFont(FontMeasurer.java:138)
  at app//org.stathissideris.ditaa.graphics.FontMeasurer.getFontFor(FontMeasurer.java:115)
  at app//org.stathissideris.ditaa.graphics.Diagram.<init>(Diagram.java:493)
  at app//org.stathissideris.ditaa.core.CommandLineConverter.convertToImage(CommandLineConverter.java:144)
  at app//org.stathissideris.ditaa.core.CommandLineConverter.doConvert(CommandLineConverter.java:110)
  at app//org.stathissideris.ditaa.core.CommandLineConverter.convert(CommandLineConverter.java:97)
  at app//io.kroki.server.service.Ditaa.convert(Ditaa.java:71)
  at app//io.kroki.server.service.Ditaa.lambda$convert$0(Ditaa.java:58)
  at app//io.kroki.server.service.Ditaa$$Lambda$255/0x0000000100497040.handle(Unknown Source)
  at app//io.vertx.core.impl.ContextImpl.lambda$null$0(ContextImpl.java:159)
  at app//io.vertx.core.impl.ContextImpl$$Lambda$206/0x00000001002e3040.handle(Unknown Source)
  at app//io.vertx.core.impl.AbstractContext.dispatch(AbstractContext.java:100)
  at app//io.vertx.core.impl.ContextImpl.lambda$executeBlocking$1(ContextImpl.java:157)
  at app//io.vertx.core.impl.ContextImpl$$Lambda$204/0x00000001002e3840.run(Unknown Source)
  at app//io.vertx.core.impl.TaskQueue.run(TaskQueue.java:76)
  at app//io.vertx.core.impl.TaskQueue$$Lambda$103/0x00000001001be440.run(Unknown Source)
  at java.base@11.0.15/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
  at java.base@11.0.15/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
  at app//io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
  at java.base@11.0.15/java.lang.Thread.run(Unknown Source)
ggrossetie commented 1 year ago

And a runtime exception thrown by Ditaa:

java.lang.RuntimeException: Cannot calculate distance of sloped edge from origin
  at org.stathissideris.ditaa.graphics.ShapeEdge.getDistanceFromOrigin(ShapeEdge.java:60)
  at org.stathissideris.ditaa.graphics.ShapeEdge.touchesWith(ShapeEdge.java:150)
  at org.stathissideris.ditaa.graphics.Diagram.separateCommonEdges(Diagram.java:746) 
  at org.stathissideris.ditaa.graphics.Diagram.<init>(Diagram.java:326)
  at org.stathissideris.ditaa.core.CommandLineConverter.convertToImage(CommandLineConverter.java:144)
  at org.stathissideris.ditaa.core.CommandLineConverter.doConvert(CommandLineConverter.java:110)
  at org.stathissideris.ditaa.core.CommandLineConverter.convert(CommandLineConverter.java:97)