Render only pages where code changed.

[ghost]

The issue: Complex diagrams containing many pages are slow to render, because it seems all pages are rendered each time. as a result: slow performance.

Possible improvement: Render only pages where code changed. (if nothing changed between two 'newpage' tag, don't re-render it, use previously rendered image.

[krasa]

edit: oh wait, nevermind. you are right.

[krasa]

So I implemented it, and also some caching of images in general. The code is quite complex now, hopefully I did not broke it too much...

[ghost]

Could you tell me how I can build the .jar needed by the IDE with theses sources?

[krasa]

• create project, import .iml file (I just pushed it)
• setup SDK - http://www.jetbrains.org/intellij/sdk/docs/basics/getting_started/setting_up_environment.html - select installed IntelliJ for which you then select JDK.
• Build | Prepore Plugin module...

[ghost]

ok trying that thanks

[ghost]

compiled OK however when in Pycharm I get NoClassDefFoundError: net/sourceforge/plantuml/cucadiagram/dot/GraphvizUtils

[krasa]

Looks like a missing library. Did it generate a zip file? And you installed that zip file? And '.PyCharmXX\config\plugins\plantuml4idea\lib' contains plantuml4idea.jar and plantuml*.jar ?

[ghost]

oops working now I was loading the .jar not the entire .zip

[ghost]

ok tested that. there is 2 bugs:

• Images are not rendered automatically. I have to click "Render PlantUML Diagram" to render an image.
• When I select a specific page in the "Select Page" Dropdown, I get "NullPointerException: null" in the Event Log. The image is not generated. If I click "Render PlantUML Diagram", the image appears.

[krasa]

stacktrace? and pycharm version?

[ghost]

OSX Pycharm 5.0.2 143.1184

null java.lang.NullPointerException at org.plantuml.idea.toolwindow.PlantUmlLabel.setup(PlantUmlLabel.java:38) at org.plantuml.idea.toolwindow.PlantUmlLabel.(PlantUmlLabel.java:28) at org.plantuml.idea.toolwindow.PlantUmlToolWindow.displayImage(PlantUmlToolWindow.java:235) at org.plantuml.idea.toolwindow.PlantUmlToolWindow.displayDiagram(PlantUmlToolWindow.java:227) at org.plantuml.idea.toolwindow.PlantUmlToolWindow$4$1.run(PlantUmlToolWindow.java:201) at com.intellij.openapi.application.impl.LaterInvocator$FlushQueue.a(LaterInvocator.java:337) at com.intellij.openapi.application.impl.LaterInvocator$FlushQueue.run(LaterInvocator.java:321) at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:311) at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:756) at java.awt.EventQueue.access$500(EventQueue.java:97) at java.awt.EventQueue$3.run(EventQueue.java:709) at java.awt.EventQueue$3.run(EventQueue.java:703) at java.security.AccessController.doPrivileged(Native Method) at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:76) at java.awt.EventQueue.dispatchEvent(EventQueue.java:726) at com.intellij.ide.IdeEventQueue.f(IdeEventQueue.java:866) at com.intellij.ide.IdeEventQueue._dispatchEvent(IdeEventQueue.java:654) at com.intellij.ide.IdeEventQueue.dispatchEvent(IdeEventQueue.java:381) at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:201) at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116) at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105) at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101) at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93) at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)

[krasa]

I was unable to reproduce it, but made some improvements, try it now please.

[ghost]

ok testing it in 5 minutes

[krasa]

BTW if included file contains a newpage, then all pages will be always rerendered (if they are selected to render). Maybe I will fix it...

[ghost]

The 2 bugs are corrected. Works well now.

However, there is only a small adjustment left to fix: If I have two .puml documents open... Say the first document has 17 pages and the second document has only 3 pages. If I select the 16th page and then switch to the second document, I get a java.lang.ArrayIndexOutOfBoundsException: 15 (possibly because the page 16 does not exist on the second document???)

java.lang.ArrayIndexOutOfBoundsException: 15 at org.plantuml.idea.rendering.RenderCacheItem.imageMissing(RenderCacheItem.java:82) at org.plantuml.idea.rendering.RenderCacheItem.renderRequired(RenderCacheItem.java:47) at org.plantuml.idea.toolwindow.PlantUmlToolWindow$3.run(PlantUmlToolWindow.java:194) at com.intellij.openapi.application.impl.LaterInvocator$FlushQueue.a(LaterInvocator.java:337) at com.intellij.openapi.application.impl.LaterInvocator$FlushQueue.run(LaterInvocator.java:321) at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:311) at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:756) at java.awt.EventQueue.access$500(EventQueue.java:97) at java.awt.EventQueue$3.run(EventQueue.java:709) at java.awt.EventQueue$3.run(EventQueue.java:703) at java.security.AccessController.doPrivileged(Native Method) at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:76) at java.awt.EventQueue.dispatchEvent(EventQueue.java:726) at com.intellij.ide.IdeEventQueue.f(IdeEventQueue.java:866) at com.intellij.ide.IdeEventQueue._dispatchEvent(IdeEventQueue.java:654) at com.intellij.ide.IdeEventQueue.dispatchEvent(IdeEventQueue.java:381) at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:201) at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116) at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105) at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101) at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93) at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)

[krasa]

So that condition I deleted was needed after all... :) Fixed

[ghost]

1 bug left: As soon as I have a Syntax Error in my file (example, while typing slowly...), I get the "syntax error?" image, and then the problem: it seems to reprocess the complete document (so it takes like 1 minute). Also when it happens, the "Select Page" dropdown resets to "All Pages".

And I got this error when it happens:

Throwable: trying to display null image. selectedPage=-1, nullPage=0, cacheItem=org.plantuml.idea.rendering.RenderCacheItem@640e6765[version=499,sourceFilePath=/,baseDir=/,zoom=100,page=4,includedFiles={},imageResult=org.plantuml.idea.rendering.RenderResult@371ea391[diagrams=[org.plantuml.idea.rendering.RenderResult$Diagram@51616848[page=4,description=(0 entities) (31 entities) (38 entities) (25 entities) (32 entities) (26 entities) (0 entities) (15 entities) (15 entities) (13 entities) (16 entities) (7 entities) (5 entities) (21 entities) (32 entities) (23 entities) (30 entities),diagramBytesLength=71471]],pages=17,renderRequest=org.plantuml.idea.rendering.RenderRequest@436841fb[baseDir=/,format=PNG,page=4,zoom=100,version=499]],imagesWithData=[null, null, null, org.plantuml.idea.util.ImageWithUrlData@54ca23bc[hasImage=true], org.plantuml.idea.util.ImageWithUrlData@168a009f[hasImage=true], null, org.plantuml.idea.util.ImageWithUrlData@4c73092e[hasImage=true], org.plantuml.idea.util.ImageWithUrlData@4d95eec2[hasImage=true], null, null, null, null, null, null, null, null, null]] java.lang.Throwable at com.intellij.openapi.diagnostic.Logger.error(Logger.java:115) at org.plantuml.idea.toolwindow.PlantUmlToolWindow.displayImage(PlantUmlToolWindow.java:274) at org.plantuml.idea.toolwindow.PlantUmlToolWindow.displayDiagram(PlantUmlToolWindow.java:263) at org.plantuml.idea.toolwindow.PlantUmlToolWindow$MyRenderCommand.postRenderOnEDT(PlantUmlToolWindow.java:242) at org.plantuml.idea.rendering.RenderCommand$1.run(RenderCommand.java:58) at com.intellij.openapi.application.impl.LaterInvocator$FlushQueue.a(LaterInvocator.java:337) at com.intellij.openapi.application.impl.LaterInvocator$FlushQueue.run(LaterInvocator.java:321) at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:311) at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:756) at java.awt.EventQueue.access$500(EventQueue.java:97) at java.awt.EventQueue$3.run(EventQueue.java:709) at java.awt.EventQueue$3.run(EventQueue.java:703) at java.security.AccessController.doPrivileged(Native Method) at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:76) at java.awt.EventQueue.dispatchEvent(EventQueue.java:726) at com.intellij.ide.IdeEventQueue.f(IdeEventQueue.java:866) at com.intellij.ide.IdeEventQueue._dispatchEvent(IdeEventQueue.java:654) at com.intellij.ide.IdeEventQueue.dispatchEvent(IdeEventQueue.java:381) at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:201) at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116) at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105) at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101) at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93) at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)

[krasa]

Reproduced and fixed.

But, wow, 1 minute? Do you use links?

@startuml
url of Class01 is [[http://google.com]]
abstract class AbstractList
@enduml

because SVG takes a lot of time, I could add a settings for disabling it...

[ghost]

no links. I still have the problem with your latest version:

1. I type an incomplete syntax, any invalid line
2. (40 seconds of 100% CPU Usage)
3. the image "..skipping 400 lines...blabla Syntax Error?" appears
4. I correct the incorrect syntax
5. (40 seconds of 100% CPU Usage)
6. The rendered image appears

My doc is 1200 lines with about 15 'newpage' in it

When I type without writing syntax errors, all is well, the image is rendered in 1 second or two, with only a little cpu spike.

[krasa]

please enable debug logs by adding '#org.plantuml' to [Help | Debug Log Settings] and send me the log.

[krasa]

I disabled links, try it.

[ghost]

here is the log idea.txt

[krasa]

Looks correct, it rendered error - which itself is crazy slow, that is a problem of plantuml itself, and then you probably fixed it back so it had nothing to render and displayed all from cache.

[krasa]

Btw what CPU do you have? Some 2 core? I would recommend disabling editor syntax checking in settings.

[ghost]

yes MBP core duo 2.26Ghz 2009

[ghost]

yup tested plantuml in Terminal : 21 seconds to tell me there is a parse error. ugh

[krasa]

You seriously need to upgrade...

Why do you even have such big document? Wouldn't it be better to split it or use some visual tool instead?

I don't see anything more to optimize now, unless I would start to save images to disk, so I can keep bigger cache... I bet your image must be killing the heap.

[ghost]

it's just 17 small pages in one document. Navigation is easy with the dropdown and so.. I love that better than to have 17 .puml files....after all what's the purpose of the dropdown if it is not to handle 17 pages

[ghost]

your plugin send content of each 'newpage' separately to plantuml right?

[krasa]

no, it takes the whole document, and I request rendering of specific pages by their index.

[ghost]

ok that's consistent with the 22 seconds behavior. Originally I thought that only the one modified 'newpage' section was sent each time, based on a checksum of that content. That would fix the 22 seconds problem

[krasa]

That could be done, but e.g. sequence diagram would look totally different. Which diagram you use?

[ghost]

Currently I use -use cases -class -component

but I will use sequence too for sure.

I don't know for now how they will be affected, I did not test yet how they behave with 'newpage'..

[ghost]

maybe the plugin could have a parameter to choose either the current "send whole+request index" rendering strategy or a new one thats sends each page's code separately to plantuml.

[krasa]

Yep, you will be able to make a comment on the start e.g.

idea.partialRender idea.disableSyntaxCheck

I saw an improvement from 30s to 3s by splitting it!

[krasa]

Implemented, you can enable it by putting on the first page idea.partialRender idea.disableSyntaxCheck as a comment with ' or whatever

I have not seen such dramatic improvements with other diagram types, apart from when syntax is broken - I will report a bug on plantuml ...

For example http://pastebin.com/qA1TwsUA

• 4,5s to render normally -> 20s syntax error image
• 6s rendering splitted by pages -> instant error rendering

[ghost]

lightning faaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaast ! Uses almost no cpu now

[ghost]

There is a small issue with scale. Here is how it is currently handled in:

1. plantuml
2. your plugin (old version)
3. your plugin (new lightning version)
4. plantuml: using scale 0.2 scales the page to 0.2
5. your plugin (old version) In order to scale 0.2 to work properly, one must add the newpage tag at top of a document
6. your plugin (new lightning version) scale 0.2 does not work at all.

[krasa]

Is it reported on plantuml tracker?

[ghost]

isn't the bug in the plugin? scale is ok if I render the page in command line with plantuml 8041

[krasa]

Hmm, I see, the problem is my plugin overrides the scale, when there is only one page.

[krasa]

fixed

[ghost]

Do you think it could remember the page it was on when switching between two .puml documents (like it already does when switching between a .puml and a non-.puml file, which is great)

Do you think it would be possible to join the title's of pages (when there is one) with the page number into the dropdown?

[krasa]

sure, do you want it to be remembered between restarts?

[ghost]

would be a nice addition

[krasa]

So plantuml made a fix, and 30s error render reduced to 100ms :-)

[ghost]

great!

[krasa]

please try the new version https://github.com/esteinberg/plantuml4idea/releases/tag/2.0.0-SNAPSHOT

[ghost]

seems to work great!