Import custom shapes / shape libraries

[nlooije]

Elevator Pitch

Importing or adding custom shapes / libraries would help with implementing software-specific shapes. This is already part of the diagrams.net ecosystem, as seen here

Motivation

I am working on a GUI for a discrete simulation package which uses a block diagram to represent the process flow with sources, machines and queues. Currently, i have a working prototype based on ipydrawio; different blocks represent the different unit operations and arrows connect to the blocks to define the routing of the parts through the process. The simulation parameters are stored in the data property of the block. It would be nice if the process flow units available in the simulation package could be defined as shapes with specific types and parameters to be set. For this a custom shape library should be importable.

Design Ideas

Below is the prototype:

Currently, the shapes are basic rhombus (Source & Exit), rectangle (Machine) and ellipse (Queue) shapes. Their data attributes are added by manually specifying in the data property through Edit > Edit data This gives the following XML:

<object label="Source" placeholders="1" Label="Source" Id="S1" Type="Source" Data="{'interArrivalTime':{'Fixed':{'mean':0.5}},'entity':'Part'}" id="2">
  <mxCell style="rhombus;whiteSpace=wrap;html=1;movable=1;resizable=1;rotatable=1;deletable=1;editable=1;connectable=1;" vertex="1" parent="1">
    <mxGeometry x="240" y="110" width="80" height="80" as="geometry"/>
  </mxCell>
</object>

All except the Data attribute are common to all required shapes, but Data contains shape-specific parameters. It would be nice if a library containing Source, Exit, Machine, Queueu (and more as needed) shapes could be loaded which each have their own shape-specific parameters, i.e.:

Shape Source:

<object label="Source" placeholders="1" Label="Source" Id="S1" Type="Source" InterArrivalTime="{'Fixed':{'mean':0.5}}" Entity="Part" id="2">
  <mxCell style="rhombus;whiteSpace=wrap;html=1;movable=1;resizable=1;rotatable=1;deletable=1;editable=1;connectable=1;" vertex="1" parent="1">
    <mxGeometry x="240" y="110" width="80" height="80" as="geometry"/>
  </mxCell>
</object>

Shape Queue:

<object label="Queue" placeholders="1" Label="Queue" Id="Q1" Type="Queue" Capacity="1" id="1">
  <mxCell style="ellipse;whiteSpace=wrap;html=1;movable=1;resizable=1;rotatable=1;deletable=1;editable=1;connectable=1;" vertex="1" parent="1">
    <mxGeometry x="240" y="110" width="80" height="80" as="geometry"/>
  </mxCell>
</object>

etc.

[bollwyvl]

This sounds awesome, need to take a bit to unspool. I have a few other things tugging on my availability during the holidays, but would love to help, especially if it's just adding an API.

I've been very interested, especially on the widget side, of we might go about inspecting the as-delivered XML for shapes to generate classes, but it's all quite challenging, as there are no schema anywhere, just what works today.

Pulling back from the content of your plugin (which, while interesting, is kinda outside of scope) what API do you actually imagine enabling your feature? On the JS side, I figured out a way to do something for templates to support the "custom diagram" view, but that only works because it's somewhat outside of the underlying API, as we must understand how to update whole documents, and there is a public iframe API for it.

At present, I've been trying really hard to stay away from mucking about too deeply in the underlying JS API, preferring the iframe stuff, as it keeps us more lightly decoupled, and at least somewhat on the documented path. Have you looked through any of the upstream docs as to how this might be accomplished?

[nlooije]

Thanks for replying and for considering this. I don't have any working knowledge of diagrams.net / drawio / ipydrawio so forgive me if i say something stupid. Having said that, i don't think this feature is very complex. The feature is essentially already present in diagrams.net as discussed in this blog: https://www.diagrams.net/blog/custom-libraries

In diagrams.net, I have defined the objects from my example and exported them to a library xml file:

<mxlibrary>[
{"xml":"dVG7DoMwDPya7ECWri20TJ3aHwhgkVQJQca8/r6BQAGpDJF8vrv4xXhshhRFLZ+2AM34nfEYrSUfmSEGrVkUqILxhEVR4B6LHidsOLNBLRAq+mOw2QdycgotsqlYMpOThcYaPH7ZFnPwWVUR4BVRdUK/lYG9Y605w2M7DY160aK0Jmsbxm+9VASvWuQT07uJXU6SmboIXeg/7QAJhtNhwl2tFKwBwtFJelWQ9IqLnzeQoEpJx5xoPC5/zm0zLljaX+GyqxVuN5mlh5N9AQ==","w":80,"h":80,"aspect":"fixed","title":"Source"},
{"xml":"dVHBDoMgDP0a7oiXnafbTjss+wLURlhwEKxT/34VdGoyDyR9r699bWFp1gw3L5262woMSy8szby1GKNmyMAYJriuWJozITg9Jq4H2SRkuZMe3vinwBYvKJEURhaTWR6SUwmODiJ+dNBBJEvpZKlx3AoXqwD3U7Q4mrkJEdq1BM690ghP6jNlelqUOIXNZJ5QGJt+wCMMhzskG68b2AbQjyTpdYUqKk5xTa5A1wr3nGwjrn+V60EomMdf4HyiBa5fEaS7n/oC","w":80,"h":80,"aspect":"fixed","title":"Queue"},
{"xml":"dVHLDoMgEPwa7gqXnqutJ0/tD6BuhAaE4Kbq35eHVk3aA8kOO7OzD8IKPVeOW1GbDhRhN8IKZwymSM8FKEVoJjvCSkJp5h+h9z/ZPGYzyx0M+ENgmhe06BmKN8GsjMkgwcVCwjVvhRxgreRMC+Moh/4pNRwFm2WE525GXNTKnYREeFjeBjz5MQm7CtTBOvdhKvUGhzD/nSA/OFRgNKBbPGWSHYrEuKQhMwGyF3j+42PC/Ve5r8MHa9MbXBe0wf0QkXq60wc=","w":80,"h":80,"aspect":"fixed","title":"Machine"},
{"xml":"dVHLEoMgDPwa7giXnqvWU0/9AtSM0IHiYFrx78tDa51pD8xks5tsEggvjW+cGOXV9qAJrwkvnbWYI+NL0JowqnrCK8IYDY+wyx+2SCwdhYMH/iiw7R06DAot2mhWJTKW4DJCxrVXmHNb1yQ5Gk646FXvpDXtcyL8PEuFcBtFF5k57BRyEk30KUKYm77AIfi/4xZfXg1YA+iWIJlVjzIrTnkjKkENEo85MWU8fCr33UOwjr/B9Rob3K+epIdPeQM=","w":80,"h":80,"aspect":"fixed","title":"Exit"}
]</mxlibrary>

Apparantly, diagrams.net requires the xml to be compressed. Using their online encode/decode tool, this can be decompressed to:

<mxlibrary>[
{"xml":"<mxGraphModel><root><mxCell id="0"/><mxCell id="1" parent="0"/><object label="" type="Source" interArrivalTime="" id="2"><mxCell style="rhombus;whiteSpace=wrap;html=1;" vertex="1" parent="1"><mxGeometry width="80" height="80" as="geometry"/></mxCell></object></root></mxGraphModel>","w":80,"h":80,"aspect":"fixed","title":"Source"},
{"xml":"<mxGraphModel><root><mxCell id="0"/><mxCell id="1" parent="0"/><object label="" type="Queue" capacity="" id="2"><mxCell style="ellipse;whiteSpace=wrap;html=1;" vertex="1" parent="1"><mxGeometry width="80" height="80" as="geometry"/></mxCell></object></root></mxGraphModel>","w":80,"h":80,"aspect":"fixed","title":"Queue"},
{"xml":"<mxGraphModel><root><mxCell id="0"/><mxCell id="1" parent="0"/><object label="" type="Machine" processingTime="" id="2"><mxCell style="whiteSpace=wrap;html=1;" vertex="1" parent="1"><mxGeometry width="80" height="80" as="geometry"/></mxCell></object></root></mxGraphModel>","w":80,"h":80,"aspect":"fixed","title":"Machine"},
{"xml":"<mxGraphModel><root><mxCell id="0"/><mxCell id="1" parent="0"/><object label="" type="Exit" id="2"><mxCell style="rhombus;whiteSpace=wrap;html=1;" vertex="1" parent="1"><mxGeometry width="80" height="80" as="geometry"/></mxCell></object></root></mxGraphModel>","w":80,"h":80,"aspect":"fixed","title":"Exit"}
]</mxlibrary>

The compressed version can be imported:

And the template objects are then shown as shapes:

For my application it would be nice if the other shape libaries (except mine) are not imported by default. Clicking More Shapes, this is possible by deselecting the standards libraries:

It should also be possible to do programmatically as stated in the above mentioned blog as passing a parameter libs=0 when importing a library by URL opens only the imported library.

In ipydrawio, similarly shape libraries can be selected/deselected: So it looks to me that part of the functionality is already present, what is missing is an implementation of importing externel library xml files.

I am not familiar with the ipydrawio code, do you link to some older (stripped down) version of diagrams.net. Would this feature be 'automagically' be present if updated to the latest version of diagrams.net/drawio?

[bollwyvl]

Happily, in this case, right at the top of that blog post (part of my frustration with their documentation approach), it has a link that contains:

?clibs=Uhttps%3A%2F%2Fjgraph.github.io%2Fdrawio-libs%2Flibs%2Ftemplates.xml

Further documented here, clibs is a semi-colon delimited list, with the U prefix meaning URL (weird).

We do expose that in our plugin schema, which is used as the basis for code generation in a couple places, including the widget... but there's a catch: the URLs appear to need to be absolute URLs, which isn't going to work very well from the kernel, which by design doesn't know anything about URLs.

While there are a bunch of ugly ways to solve this from the python side, and one decent (if involved) way to solve this from the typescript side, there's a chance, however, this can be done via a data:// or blob:// URL. I'll take a look.

[bollwyvl]

Here's a hack I could get working from the python side.

https://mybinder.org/v2/gh/bollwyvl/jupyterlab-drawio/gh-80-widget-custom-shapes?urlpath=lab%2Ftree%2Fdocs%2Ftutorials%2Fworking-with-custom-libraries%2Findex.ipynb

I wouldn't mind shipping this change, but don't have a clear vision of how to make a useful, reliable API out of this... it seems like it might be good enough for your case, but whatever we do sure as shooting isn't going to be portable to other UIs, or even very resistant to change between versions.

[nlooije]

@bollwyvl This look like (at least for my use case) you are basically there! That's great!

Some feedback:

• There is a trailing <!-- /my library --> in the xml encoding. It shouldn't be there correct? It seems to be messing with the label of the library:
• Follow up to the above point, how can we set the label / name of the library? I am not sure if it is possible, can't find it in diagrams.net or in docs.
• Additionally setting libs=0 in the url_params disables all other libraries except the custom library and one other Trees. This is almost how i would want it, but how to get rid of the Trees library?

Sure it is slightly hacky, but i don't mind as long as it works, if needed i'll throw it in a function somewhere or perhaps subclass the Diagram class. I don't understand exactly why it wouldn't be portable to other UIs (which?) or resistant to changes between versions (which part would break?). Considering this feature may not gather a whole lot of attention from users, maybe you should keep things practical and ship it as is?

Thanks for the implementation though, one step closer to a neat GUI interface

[bollwyvl]

• There is a trailing in the xml encoding.

Yep!

• It shouldn't be there correct?

It was the best I could do to not be mxlibrary>, which seemed strictly worse.

• It seems to be messing with the label of the library: [image: image] https://user-images.githubusercontent.com/3252193/147298784-7f358f12-0e32-420f-83c4-5a5786a81912.png

Right: for non-builtins, it calculates the name from the last / .

https://github.com/jgraph/drawio/blob/v16.0.3/src/main/webapp/js/diagramly/UrlLibrary.js#L18

- https://user-images.githubusercontent.com/3252193/147298784-7f358f12-0e32-420f-83c4-5a5786a81912.png

• Follow up to the above point, how can we set the label / name of the library? I am not sure if it is possible, can't find it in diagrams.net or in docs.

I couldn't really find a way other than the comment hack. it's going to go through that code, and I don't see a way to change the title from the call site of that constructor.

• Additionally setting libs=0 in the url_params disables all other libraries except the custom library and one other Trees. This is almost how i would want it, but how to get rid of the Trees library?

I don't know! Probably with tinkering with more of the url_params, it might be possible. I figured by getting to where one can mess with it interactively on e.g. binder, I didn't have to anticipate exactly what you needed.

Sure it is slightly hacky, but i don't mind as long as it works, if needed i'll throw it in a function somewhere or perhaps subclass the Diagram class. I don't understand exactly why it wouldn't be portable to other UIs (which?)

I mean more from the "using it like a drawing program" perspective. Existing shapes would probably work, but users would have to have access to the xml file to upload, and manually import your libraries, to create more of them.

or resistant to changes between versions (which part would break?).

It's a fairly hacky patch, and the upstream is unlikely to take it, having many other fish to fry. I'll work this tutorial up to where it works and put it under robot testing, but there's the risk that at some point it might no longer be feasible, and there won't be much I can do about it.

Considering this feature may not gather a whole lot of attention from users, maybe you should keep things practical and ship it as is?

Don't have a ton of users/downstreams that aren't myself and projects I work on, at least ones that ask for (reasonable) things/give feedback/make contributions... So yeah, provided it works with the 16.x line, I'll probably skip it before the end of the year, unless we figure out a way to do it more cleanly.

Thanks for the implementation though, one step closer to a neat GUI interface

Right, I'm motivated to help you succeed without having to do too much crazy stuff! Of course, if you did wade into labextension land, you could probably get much closer to exactly what you want, but that's a significant complexity increase, and you, too, have other fish to fry/simulate the frying of.

[nlooije]

Ok, now i understand the point of the trailing closing comment. I have noticed that diagrams.net custom libaries use the file name that was imported as label, e.g. example.xml become example, as far as i can tell the label isn't specified in xml. It isn't very important (at least to me), as long as i know a way of specifying some sort of label, which i do now.