Optimize loading time by analyzing the codebase and dependencies

[sakirtemel]

https://github.com/plantuml/plantuml/issues/803#issuecomment-1404080146

[arnaudroques]

To speed up starting time, I think we will end up with having a specific plantuml.jar file for plantuml-wasm because there are many things that is useless in the context of wasm. We are going to work on this, but please be patient :-)

On short term, you can remove all the following jars, because they are not used:

Used for elk layout:

• elk-full.jar

Used for PDF export:

• avalon-framework-4.2.0.jar
• batik-all-1.7.jar
• commons-io-1.3.1.jar
• commons-logging-1.0.4.jar
• fop.jar
• xml-apis-ext-1.3.04.jar
• xmlgraphics-commons-1.4.jar
• jlatexmath-minimal-1.0.3.jar
• jlm_cyrillic.jar
• jlm_greek.jar

Used for VizJS

• vizjs.jar
• j2v8_win32_x86_64-3.1.6.jar
• j2v8_linux_x86_64-3.1.6.jar
• j2v8_macosx_x86_64-3.1.6.jar

Used for ditaa:

• ditaa0_9.jar

Even is there are some complain in the log, as long as it is working, that's fine. Any tough ?

[sakirtemel]

To speed up starting time, I think we will end up with having a specific plantuml.jar file for plantuml-wasm because there are many things that is useless in the context of wasm. We are going to work on this, but please be patient :-)

that looks like it. even not only for speeding up the starting time, but making it let's say more web-friendly(rendering speed, dependenies etc). I haven't read this one yet, but when I skimmed through it, they're kind a explaining having Java on the frontend as a choice, not for portability : https://plantuml.github.io/plantuml-wasm/#example=examples/community/cloudflare_linode_firewall.puml . I assume similar patterns are applied here

On short term, you can remove all the following jars, because they are not used:

Thank you for detailedly explaining what's used as dependencies, it was a bit missing for me. I tried to compile without any dependency, however the size was the same ~25mb. I'll do that again, also I'm going to add the compile log, so you'll be able to see what files are being processed

[sakirtemel]

As @fabiospampinato mentioned https://github.com/plantuml/plantuml/issues/803#issuecomment-1404452441 , we can aim for a single, small javascript file as an ultimate goal :)

[sakirtemel]

file sizes in compiled-jars

total 123M

4.0K    jlm_cyrillic.jar.js
4.0K    jlm_greek.jar.js
20K     vizjs.jar.js
40K     commons-logging-1.0.4.jar
84K     commons-io-1.3.1.jar
84K     xml-apis-ext-1.3.04.jar
88K     avalon-framework-4.2.0.jar
136K    commons-logging-1.0.4.jar.js
152K    xml-apis-ext-1.3.04.jar.js
184K    ditaa0_9.jar
184K    jlm_cyrillic.jar
184K    jlm_greek.jar
256K    j2v8_linux_x86_64-3.1.6.jar.js
256K    j2v8_macosx_x86_64-3.1.6.jar.js
256K    j2v8_win32_x86_64-3.1.6.jar.js
296K    avalon-framework-4.2.0.jar.js
352K    commons-io-1.3.1.jar.js
556K    xmlgraphics-commons-1.4.jar
568K    vizjs.jar
628K    jlatexmath-minimal-1.0.3.jar
1.1M    ditaa0_9.jar.js
1.4M    jlatexmath-minimal-1.0.3.jar.js
2.1M    xmlgraphics-commons-1.4.jar.js
2.8M    j2v8_linux_x86_64-3.1.6.jar
3.2M    batik-all-1.7.jar
3.2M    j2v8_win32_x86_64-3.1.6.jar
4.2M    fop.jar
5.8M    elk-full.jar
7.4M    j2v8_macosx_x86_64-3.1.6.jar
11M     plantuml-1.2023.0.jar
15M     batik-all-1.7.jar.js
16M     fop.jar.js
23M     elk-full.jar.js
25M     plantuml-1.2023.0.jar.js

file sizes in compiled-lite-jars

11M     plantuml-1.2023.0.jar
25M     plantuml-1.2023.0.jar.js

Output file for compiled-lite-jars:

compile_output.txt

[sakirtemel]

With only using compiled-lite-jars (only plantuml), just a single js, it still works, however it gives these errors

You can try with this link: https://plantuml.github.io/plantuml-wasm/#lite=true

it looks like it's a bit faster when initializing: https://plantuml.github.io/plantuml-wasm/#example=examples/community/cloudflare_linode_firewall.puml&lite=true

the fastest I was able to get is ~5 seconds, it's when I refresh the page(probably the jar/js is cached),, without lite it's ~8 seconds

for the first load with lite=true, it's ~11 seconds, without lite, ~15-16 seconds

(by the way, I added console log for each rendering, it can always be useful)

[arnaudroques]

So we quickly&dirty made a specific version of PlantUML for wasm:

http://beta.plantuml.net/plantuml-wasm.jar

We've removed many things that are not necessary for WASM and we've adapted the run. You must now provide 3 arguments:

1) The format (should be "png" or "svg") 2) The path of the generated file ( "result.png" for example) 3) The actual text of the diagram

For example:

java -jar plantuml-wasm.jar svg out.svg "The actual text of the diagram with carriage return"

If I understand correctly, you should adapt the .html file from:

cheerpjRunMain("net.sourceforge.plantuml.Run", "/app" + pathname + jarsPath + "/plantuml-1.2023.0.jar", "-Playout=smetana", "-o", "/files/", "/str/input.puml").... to: cheerpjRunMain("net.sourceforge.plantuml.wasm.Run", "/app" + pathname + jarsPath + "/plantuml-wasm.jar", "png", "result.png", editor.getValue())....

The big question is whether carriage return (in editor.getValue()) would be correctly managed. We are sorry we could not test it. Tell us, and also tell us if it speeds up the starting time and the diagram generation.

Thanks!

[sakirtemel]

6.6M    plantuml-wasm.jar
21M     plantuml-wasm.jar.js

compile_output.txt

https://plantuml.github.io/plantuml-wasm/#beta=true

it does generate the image, however somehow I can't get if the jar exited or not. I was able to see the image if I load it manually. I'll troubleshoot what's going wrong

also added carriage return at the end if missing

[arnaudroques]

it does generate the image, however somehow I can't get if the jar exited or not.

Ok, the issue is probably on our side: we are going to build another jar

also added carriage return at the end if missing

Ok, we will pad a carriage return at the end

[arnaudroques]

So you can try the new jar (same url http://beta.plantuml.net/plantuml-wasm.jar )

[sakirtemel]

So you can try the new jar (same url http://beta.plantuml.net/plantuml-wasm.jar )

this gives me the same file? at least my loca git is not saying that the file is modified :)

[arnaudroques]

Sorry, let's try http://beta.plantuml.net/plantuml-wasm-v3.jar :-)

[sakirtemel]

Well, I updated it. seems like exitcode still returns undefined, I don't know if it's the reason, but I can't have the javascript promise to be resolved for the first time.

so, here's what is here:

when you open it, it doesn't initialize/render for the first time: https://plantuml.github.io/plantuml-wasm/#beta=true

however, if you type something on the editor, the same function works fine

[sakirtemel]

ok, I don't know why, but it started to work.

~4 seconds 🎉

[sakirtemel]

added preloading runtime resources: https://github.com/plantuml/plantuml-wasm/commit/c6197b961eac45d928948e90902954b66d364807

less than ~3 second on reload first opening less than ~8 second

[arnaudroques]

After reading https://docs.leaningtech.com/cheerpj/Runtime-API.html here is yet another version http://beta.plantuml.net/plantuml-wasm-v4.jar

In this version, there is still the net.sourceforge.plantuml.wasm.Run class but we have also added a new util class:

package net.sourceforge.plantuml.wasm;
public class Utils {
    public static int convertPng(String pathOut, String text) throws IOException {
    ...
    }

    public static int convertSvg(String pathOut, String text) throws IOException {
    ...
    }

So according to the doc, you can have:

/* Equivalent Java code: int returnVal = net.sourceforge.plantuml.wasm.Utils.convertPng("result.png", "@startuml\nalice->bob\n@enduml); */
var returnVal = cjCall("net.sourceforge.plantuml.wasm.Utils", "convertPng", "result.png", editor.getValue());

You can test returnVal (0 is everything is fine, 1 if error).

It may be more convenient than using cheerpjRunMain and will probably provide in the future a better interaction between Java and JS.

Tell us if it's working fine.

We are going to have a look at standard library integration :-)

[sakirtemel]

seems a bit faster: https://github.com/plantuml/plantuml-wasm/commit/fa1a9e812567e1eca7b78dc7331f250e009bed34

however I'm still calling cheerpjRunMain for the first time, otherwise it doesn't load the package. Maybe there's a way to only call cjcall.

Is there a method that I can call which gives response immediately? Like checking the version etc?

[sakirtemel]

Re-rendering simple diagrams takes like ~500 ms, it's so fun now!

https://plantuml.github.io/plantuml-wasm/#example=examples/basic/sequence_diagram.puml&beta=true

[arnaudroques]

Is there a method that I can call which gives response immediately? Like checking the version etc?

Yes, in class net.sourceforge.plantuml.version.Version, there are several very simple static methods:

public static int versionPatched();
public static String versionString();
public static String fullDescription();
public static String compileTimeString();

It's strange that you have to call cheerpjRunMain first: I don't think that we (in PlantUML I mean) are doing any initialization here. We are going to write an empty main() method that you will be able to call first. Does it sound good?

[sakirtemel]

It's strange that you have to call cheerpjRunMain first: I don't think that we (in PlantUML I mean) are doing any initialization here. We are going to write an empty main() method that you will be able to call first. Does it sound good?

Yes, let's give it a try. It's a bit hacky solution, but it works :D it would be nice if this main method returns/prints the version actually?

[arnaudroques]

So here is a new version http://beta.plantuml.net/plantuml-wasm-v6.jar

In this version, there is a net.sourceforge.plantuml.wasm.RunEmpty class with a main() method which does nothing else but printing the actual version (twice, in standard error and in standard output). Does the hacky solution work?

(NB: In some future, we are going to prepare a dedicated git branch for this plantuml-wasm version so that everybody could see the code, but right now we are preparing next release for the incoming days, so we wait until there)

[sakirtemel]

(NB: In some future, we are going to prepare a dedicated git branch for this plantuml-wasm version so that everybody could see the code, but right now we are preparing next release for the incoming days, so we wait until there)

once the branch is there, I can make here a github workflow that (semi?)automatically fetches the latest jar and recompiles everything in a pull request

[sakirtemel]

So here is a new version http://beta.plantuml.net/plantuml-wasm-v6.jar

this jar didn't compile at all for some reason, is there any error in it?

[arnaudroques]

this jar didn't compile at all for some reason, is there any error in it?

Yes: we forget to export the .class files :-)

So here is http://beta.plantuml.net/plantuml-wasm-v7.jar Sorry about all that, can't wait until we have a proper CI/CD

This v7 is large because it has the full standard library embedded. We really need to test this. If the size is an issue, we'll create a v8 without the standard library.

[sakirtemel]

Sorry about all that, can't wait until we have a proper CI/CD

No problem at all, but I created a small script anyway, you can see how it works here

https://github.com/plantuml/plantuml-wasm/pull/19

so, whenever we want to update the jar, just upload it into a new branch with the same name (can also be done via browser), and create a PR. It automatically recompiles and pushes the recompiled version to the PR branch.

Since we're going to still iterate until we make it stabilized, it's good to have some automation :)

[sakirtemel]

This v7 is large because it has the full standard library embedded. We really need to test this. If the size is an issue, we'll create a v8 without the standard library.

Yes, let's test this. Since it's beta, no-one is affected by the size/experiments anyway

[sakirtemel]

In this version, there is a net.sourceforge.plantuml.wasm.RunEmpty class with a main() method which does nothing else but printing the actual version (twice, in standard error and in standard output). Does the hacky solution work?

🎉 worked

[sakirtemel]

@arnaudroques stdlib worked, too : https://github.com/plantuml/plantuml-wasm/issues/3#issuecomment-1406936878 🎉

[arnaudroques]

once the branch is there, I can make here a github workflow that (semi?)automatically fetches the latest jar and recompiles everything in a pull request

So the branch is here https://github.com/plantuml/plantuml/tree/wasm I'm uncomfortable with GitHub action so I've deleted it in this branch... Could you still create a github workflow to build everything back (I mean, even in the wasm branch)

Note that stdlib will also be broken, but that's because I want to try another solution :-)

Thanks again

[sakirtemel]

@arnaudroques created a semi-automated flow

https://github.com/plantuml/plantuml-wasm/actions/workflows/build-plantuml-from-source.yml

when you click on run workflow, it automatically clones the plantuml repo, wasm branch and runs gradle here. This flow does not include releasing the jar, as I assume it's not needed at al. With this way, I didn't need to touch the plantuml repositories' flows

Works good?

[sakirtemel]

there's a small issue I'm troubleshooting, somehow the other workflow that needs to run on the workflow is not running

it should be fixed now with this run: https://github.com/plantuml/plantuml-wasm/actions/runs/4038476538

[arnaudroques]

Works good?

Sounds good!

there's a small issue I'm troubleshooting, somehow the other workflow that needs to run on the workflow is not running

Sure, take your time: no pressure here. BTW, there is no need to have me as reviewer here: I don't understand what I am supposed to review :-D

[sakirtemel]

Ah, the https://github.com/marketplace/actions/create-pull-request is automatically assigning you probably :)

Works good?

Sounds good!

there's a small issue I'm troubleshooting, somehow the other workflow that needs to run on the workflow is not running

Sure, take your time: no pressure here. BTW, there is no need to have me as reviewer here: I don't understand what I am supposed to review :-D

— Reply to this email directly, view it on GitHub https://github.com/plantuml/plantuml-wasm/issues/5#issuecomment-1407759460, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAHFKUGEIVFGICTUUXYVHALWU3FSDANCNFSM6AAAAAAUG7HFWA . You are receiving this because you authored the thread.Message ID: @.***>

--

Regards,

Sakir Temel

[sakirtemel]

@arnaudroques ok, all works now.

https://plantuml.github.io/plantuml-wasm/#beta=true

I can also see that the stdlib stopped working, and normal render still works

[arnaudroques]

I can also see that the stdlib stopped working, and normal render still works

Yes, that's normal. On my side, I've just made a modification in the java source code

Rather than embedded ZIP file in the jar, all .puml file from the standard library are expected somewhere on the webserver. Not sure where exactly, the browser log will probably tell us :-)

[arnaudroques]

So I just launch the workflow!

[arnaudroques]

Rather than embedded ZIP file in the jar

Reading the ZIP file in Java/JS is probably much longer than getting directly the right file from the webserver, even if it means you make many requests to the server (one per file). I think it's better: we'll see...

[arnaudroques]

Ok, it compiles! That's great!

Now, what's not clear to me is whether https://plantuml.github.io/plantuml-wasm/ is automatically updated or not ? And I see from the log browser that it still loads many jars like ditaa0_9.jar and so on. This really slow down the page load.

[sakirtemel]

First you need to merge the PR, only the main branch is served(and it takes ~4 minutes).

And after that, the build from source is only used with the beta flag https://plantuml.github.io/plantuml-wasm/#beta=true

[sakirtemel]

Is that possible that you can run it with a path? Probably you should be able to get the jar's execution path, it should be the same, for this instance it's /app/plantuml-wasm/compiled-beta-jars/ , that's where the jar file is, and that's where we can put our files to

so, full path will be for ex. /app/plantuml-wasm/compiled-beta-jars/stdlib/awslib14/AWSCommon.puml

[sakirtemel]

also btw, normal rendering(simple sequence diagram) got broken :)

[arnaudroques]

also btw, normal rendering(simple sequence diagram) got broken :)

Are you sure? It seems to work for me.

Is that possible that you can run it with a path?

~~Yes... and no. :-) I suggest we go step by step. First I need to have all *.puml file to be server somewhere on the server. So I need to have https://plantuml.github.io/anything/you/would/like/stdlib/awslib14/AWSCommon.puml to be served.~~

So I've just commit all those files hopping it's fine for you. Now I really need those files to be served by https://plantuml.github.io somewhere. EDIT: They are here https://plantuml.github.io/plantuml-wasm/stdlib/awslib14/AWSCommon.puml

[arnaudroques]

Ok, file seems to be there https://plantuml.github.io/plantuml-wasm/stdlib/awslib14/AWSCommon.puml

So I've patched the Java source.

The workflow is running... maybe it will work?

[arnaudroques]

maybe it will work?

It looks like it's working! I let you double check that it's also fine on your side (I had to clear the cache several time...)

If it's fine on your side, I think we can remove the regular render() function. Note also that we don't need a carriage return at the end anymore (well, I think :-)

[sakirtemel]

@arnaudroques just tested, works like charm! congrats 💯

we may now make the "beta" as main and start tidying up the code. a couple of notes for the beta part:

- right now, on each key stroke, it tries to download the files again, unfortunately it's not caching them, we should find a solution for that My bad, I enabled Disable Cache before :D I confirm that consecutive calls are directly responded by cache, so it doesn't go to the server

• files are under root stdlib now, I'll think about it tomorrow where exactly they should be, because if we want to make it as a library later on, it might be problematic
• so, we don't want to serve these files as zip files, right?
• this beta version has smetana by default and only as far as I understand. Is it stable and has all the necessary features? Because I see differences in the following diagram for instance:

-- https://plantuml.github.io/plantuml-wasm/#example=examples/community/cloudflare_linode_firewall.puml

-- https://plantuml.github.io/plantuml-wasm/#example=examples/community/cloudflare_linode_firewall.puml&beta=true

[sakirtemel]

I clean the things up, no more beta, only main version :)

https://github.com/plantuml/plantuml-wasm/blob/main/assets/js/plantuml.js

I believe we can iterate the issues on the main step by step. Current version works, it's fast enough, has everything. It just has glitches :)

[sakirtemel]

by the way, about the diagram rendering issue above, the original should look like this

however, it looks like this : https://plantuml.github.io/plantuml-wasm/#example=examples/community/cloudflare_linode_firewall.puml . no left to right, spaces are considered as new nodes somehow, some notes are disappearing

[sakirtemel]

I just cleared the cache and made some tests, seems like we optimized enough :)

on refresh:

jar file is ~6.4mb, js file is 20M, however it's transferred with gzip automatically, so it becomes ~3.2MB

[arnaudroques]

• files are under root stdlib now, I'll think about it tomorrow where exactly they should be, because if we want to make it as a library later on, it might be problematic

A new issue has been opened here https://github.com/plantuml/plantuml-wasm/issues/28

• so, we don't want to serve these files as zip files, right?

Right, we don't want to serve them any more

• this beta version has smetana by default and only as far as I understand. Is it stable and has all the necessary features? Because I see differences in the following diagram for instance:

It it sable but it's not complete. Right now, this was not a high priority for us because it was just a work around for people that could not install graphviz.

But everything has changed: since this is mandatory for plantuml-wasm, we are going back to it. It's going to take time, though, but I think it's advanced enough for a first version.

jar file is ~6.4mb, js file is 20M, however it's transferred with gzip automatically, so it becomes ~3.2MB

Well, there are still some dead code here. So we will continue to work on this. I think this will not reduce the transfer time, but it may improve the speed rendering.

It was a lot of fun to make up this thread, but I suggest that we close it now (since it's somehow complete). Let's have fun in all other threads! :-)