Handle collapsed sub-processes

[MaxTru]

This is an epic issue that may be broken down into multiple tasks if needed

What should we do?

In Q2/2021 we want to have an initial implementation allowing collapsable sub-processes in bpmn-js.

Various design and technical solution approaches exists. Hence we need to:

Disclaimer: this is not completed and will be adjusted while working on this epic

• [x] Test out and decide on a conceptual (in terms of user experience) solution to implement the requirement. As part of this, also define the concrete key requirements (e.g., support drill-down via tabs, keep undo/redo history across levels, ...)
• [x] Design and agree a technical solution to implement the requirement
• [x] Implement Viewer Components
  • [x] https://github.com/bpmn-io/diagram-js/issues/560
  • [x] #1472
  • [x] #1474
  • [x] #1483
  • [ ] Implement Ghost elements #1484
  • [x] Ensure compatibility with wellknown existing extensions (cf. https://github.com/bpmn-io/internal-docs/issues/340)
  • [x] Correct resize handles https://github.com/bpmn-io/bpmn-js/issues/1520
  • [x] #1522
• [x] Implement Modeler Components
  • [x] Basic editing behavior
    • [x] #1536
    • [x] Attach drill down overlay to newly created collapsed sub-process
  • [x] #1539
    • [x] Add / remove planes accordingly
    • [x] Move elements into / out of planes
    • [x] Reconnect message flows
  • [x] Handle collapsed sub-process deletion
    • [x] Delete nested elements contained in plane
    • [x] Navigate out of plane if it gets deleted
  • [x] #1575
    • [x] update Breadcrumbs
  • [x] #1556
  • [x] diagram-js: Add make it possible to rename a plane (needed to maintain our plane#name -> element.id) relationship in the presence of updating the ID property of an element
  • [x] #1546
  • [x] Moving over breadcrumbs blows up editor https://github.com/bpmn-io/bpmn-js/issues/1511
• [ ] ~Implement Ghost element modeling~
  • [ ] ~Update Secondary Object/Fake Connections when Primary is changed~
  • [ ] ~Moving the expanded process on the implicit root should not be possible/have intuitive behavior~
  • [ ] ~Decide how resizing of the expanded Process is handled~
  • [ ] ~Handle secondary boundary event element updates~

Why should we do it?

Several requests to have this from customer side exist and we commited to deliver a solution. Also see a variety of existing related issues (list not complete):

• https://github.com/bpmn-io/bpmn-js/issues/1425
• https://github.com/bpmn-io/bpmn-js/issues/945
• (related, but focussed on visual overlap when expanding) https://github.com/camunda/camunda-modeler/issues/2004
• (related, but focussed on visual overlap when expanding) https://github.com/camunda/camunda-modeler/issues/1343

Requirements

• I can click on the plus symbol or directly interact with the diagram element in some other way
• I can navigate back to the previous level of the diagram hierarchy
• I can have multiple diagrams within the same BPMN model open at the same time, e.g. in tabs
• I can create a new diagram for a collapsed embedded sub-process to model its contents
• A bonus would be some kind of tree representation of the diagram hierarchy

Resources

Tracking the Rollout through this issue: https://github.com/bpmn-io/internal-docs/issues/340

[marstamm]

List of Problems we need to solve: https://github.com/marstamm/embedded-sub-process-prototype/

Prototypes can be tried out here: https://marstamm.github.io/embedded-sub-process-prototype/

[marstamm]

I shared the current prototypes with @andreasgeier and @volkergersabeck. The key results of the discussion are:

• We want a solution that works on the canvas as a sensible default when integrating the viewer. We can expand on it for modeling/ alternative navigation depending on the tool.
  • To validate this, I will create a Cockpit-Migration mock-up where the canvas prototypes can be tested.
• In the current prototypes, it is not clear when we are in a sub-process. I will investigate how it would look like with:
  • Rendered borders of the sub-process (zoom-in)
  • Split-view

We need to clear up some fundamental questions:

• If we go for Breadcrumbs/Tree views, do we want to show the execution hierarchy (incl. expanded sub-processes) or go for collapsed only?
• How do we handle message flows that go inside a collapsed sub-process? Do we allow modelling across Sub-Process Borders? (Probably not)
• Check what elements are allowed in a collapsed sub-process --> BPMN Spec

Next Steps:

• Create concepts as described above
• Check spec for limitations/allowed elements

[marstamm]

Create concepts as described above

Overview of all Prototypes and the outcome so far: https://docs.google.com/spreadsheets/d/1tbMYnNGHUFw42-aXuF_u8nge5hetUd8auaPKdYvGG8c/edit?usp=sharing

Check spec for limitations/allowed elements

A sub-process can contain all elements a root-process can, namely Activities, Events, Gateways, Sequenz Flows and common elements such as Messages. This includes event-based sub-processes in the sub-process.

Pools and Participants can not be part of a Process.

[marstamm]

User Tests

We are using the prototypes for user tests. Part one covers navigating the Diagrams, the second part covers borders/display of child processes.

Notes are shared in this miro board (WIP) https://miro.com/app/board/o9J_lABH5i8=/

[marstamm]

Technical Prototype preparations

Outcomes:

• Use a pragmatic approach to rendering:
  • Render everything and hide elements not displayed
  • Use layers in Canvas
  • Check DI documentation and Signavio solution to layers
• "Ghost" sequence flows should not be part of the XML
• Undo/Redo will need limitations, keep same command stack
• Technical Spearing Partner will be @philippfromme

Miro Board: https://miro.com/app/board/o9J_l-lFgjQ=/

[marstamm]

Breakdown from the first Prototype

The Prototype is on the branch sub-process-prototype on bpmn-js and diagram-js To achieve the minimal scope (Must-Have on Miro), we need to adjust the following:

BPMN Structure

• Use Multiple Diagrams in one file
• Allow multiple DI representations for a single bpmn element

Diagram-JS

• Expand Canvas Layers to support one root element per layer.
• Have a clear API for hiding/showing layers.
• Make overlays Layer-aware.

BPMN-JS

Import

• Import all present diagrams
• create map diagram -> di for each object
• Allow rebinding of di during execution

Modeling

• On diagram Change, rebind the element di according to the di map.
• Drill-down on empty process creates new diProcess and Plane, maps the planeDi to the subProcess element.
• di is not removed automatically when removing the activity -> recursively remove all referenced processes
• Copy-Pasting works out of the box.

Saving

• no changes needed -> di is correctly updated when the di binding is changed when switching layers

Navigation Features

• Overlays and Breadcrumbs

[marstamm]

Team alignment

Synced with the designers of Product Teams to understand where Problems could pop up

Optimize

In general, the integration with Optimize seems to be straight-forward

• Use Heatmaps - needs retooling to work for multiple layers
• Tooltips are always shown above of the activity - no overlap
• Diagrams are also shown in small preview windows --> make sure Breadcrumbs are not to big

Features:

• Select Sub-Process on Report creation: API to open sub-process

Operate

The main issue is the overlap with badges in the lower-right corner. This is a problem when a Sub-Process has incidents. Possible solutions:

• Slotting system: in case both are present, show the open-overlay under the badge
• (To be confirmed:) Use the top-right corner: this corner is used for "Finished" (end events), might be available for subProcesses.

Cockpit

Similar issue as Operate.

• Completed badges are shown in the lower-left. Cockpit already has a homebrew system that shows badges next to eachother when there are multiple in the same spot.
• Code freeze for 7.16 Platform will be early this year (Mid-September)

[marstamm]

Miro Link for Kickoff: https://miro.com/app/board/o9J_l7NuUV4=/

[andreasgeier]

UX review (intermediate state)

Attendees: @marstamm @andreasgeier

• Design of the drill-down button Is aligned with Cawemo (color, size, border radius). Comment Andi: Okay for now. But when the design spec of the Visual IDE is implemented, this will be the guideline for our libraries. Other tools have to solve issues by overriding the CSS.
• Position of the drill-down button Is aligned with Cawemo and equal to the link button for call activities. There is no positioning conflict because the link and the drill-down button will not appear on the same elements.
• Appearance of the drill-down button Aligned with the link button for call activities. It Will not be visible on empty subprocesses but appears on selection. By navigation inside the subprocess, a new plane is created and the subprocess gets "content". If a subprocess has content, the drill-down button on its parent stays visible permanently. Difference between viewer and editor: drill-down button won't appear on select of empty subprocesses Comment Andi: It's not preferred that the navigation alone creates an "is-edited" state. At a later point, we should include the deletion of a plane if its content is empty after visiting.
• Navigation behavior When creating a new plane, it will be shown like a new diagram: showing an initial start event, zoom factor 100%, diagram position 0, 0 When opening a plane for the first time during a user session, it will be shown like the first opening of the diagram: zoom 100%, position 0, 0 Every plane preserves its state (zoom, position) while switching between them (within a user session)
• Breadcrumb Will appear in planes but is not visible on the root parent.
• Breadcrumb position The position is in conflict with the token simulation. If compatibility is a requirement, it needs to be solved. Idea: top centered position for bread crumb.
• Integration with other apps We are aware of conflicts and open questions (operate overlays, token migration in cockpit, heat maps in optimize, etc.). This will not be tackled right now.

Action points

• @marstamm Clarify with PM if compatibility with token simulation is a (new) requirement.

[andreasgeier]

@marstamm We did not talk about the scalability of the breadcrumb. Do you already have a solution for long breadcrumbs that would exceed the viewport width? (this also includes long labels of subprocesses)

[marstamm]

No, I have not thought about long breadcrumbs yet. One possibility is to use ellipsis and only show the first and last x Layers in those cases. E.g. Root > ... > Layer 35 > Layer 36 > Layer 37

We could also think of a maximum width and text-overflow: ellipsis; for individual Label Names (and especially IDs if no name is set).

[nikku]

The position is in conflict with the token simulation. If compatibility is a requirement, it needs to be solved. Idea: top centered position for bread crumb.

Should be fixed in token simulation. We should not account for extensions and how they do things in the core.

[marstamm]

In addition to this, we plan to move the breadcrumbs to the expanded sub-process once we display it, so the conflict might only exist on a specific version or not at all

[andreasgeier]

@marstamm Let's add your proposals for scalability to the list of todos.

• Cut long labels at x characters or at a width of x px and show ellipsis. Include a title tag to provide the full name of labels.
• Use ellipses for long breadcrumbs (see Cawemo example)

[andreasgeier]

Quick sync @marstamm and @andreasgeier

New challenges because of the "ghost" borders:

• keep compatibility with Signavio, they don't use subprocess shapes on planes
• positioning (top, left) on planes because subprocess shape is "fake" and not considered
• to be ensured: same modeling behavior like on the root (auto-resize, no placement outside)
• to be decided: prop panel for subprocesses and boundary events on planes?

[nikku]

Sync results with @marstamm:

In addition to unsolved UX issues Ghost elements cause a number of technical issues, both implementation wise and when integrating with the new feature in other applications.

:arrow_right: We'll make a conscious decision whether to support them. My proposal is to drop them for the time being.

[nikku]

Results of a minor modeling without ghost elements spike:

We are able to support basic modeling operations using the abstractions we got. What we need to implement is nothing out of the ordinary, which is a good sign. Still, a long walk to go :handbag:.

---

Basic findings:

• On the Canvas we need the ability to rename planes (to support our update id behavior and maintain the plane#name -> element#id relationship
• Implementation of plane behavior can be done using the existing behavior abstractions we got

[nikku]

Results of my spike available on the subprocess-modeling branch.

[marstamm]

Result of Today's meeting:

• We will not ship ghost elements for now
• the costs do not justify the benefits
• The customer just requested basic drilldown/collapsed process support

[kurtsys-howest]

Just wondering, is there any target for support modeling of collapsed subprocesses. Issue #1553 seems to be merged, but in the demo environment ( https://demo.bpmn.io/ ), I can't seem to find it. (btw, great job! It would be such an amazing thing to have, the subprocess collapse/expand support)

[marstamm]

Hi @kurtsys-howest ,

Thanks for your interest in this feature :smile: Please see the main Comment for the current Progress. Some things are still work in Progress, such as Copy/Pasting.

Once everything is done (and this issue is closed), we will release a new major version of bpmn-js, which we will then also be available on the demo Site.

[marstamm]

Released as v9.0.0

[diggy128]

Any clue when to expect a new stable release of camunda-modeler to support this?

[MaxTru]

Any clue when to expect a new stable release of camunda-modeler to support this?

12.April 2022 is the plan. You can already use it in the nightly

[diggy128]

We only use stable but April's 12th is just a week away. Thank you so much so the prompt response!