Build a first version of the new React-based DMN Editor with ~90% feature parity with the current GWT-based one

[tiagobento]

A screenshot of the new React-based DMN Editor.

---

As work on #439 continues, I thought changing the task granularity would help. We'll start with a first "base version" of the new DMN Editor, and we'll continue evolving it, eventually replacing the old one. For this first "base version" to be ready, many little things need to be addressed. As the list is long, and some things might be postponed, I figured we could use this issue to track stuff that goes into this first "base version", and things that will be addressed after it can have their own separate issues.

The first "base version" is being developed at https://github.com/tiagobento/kie-tools/tree/dmn-editor at packages/dmn-editor. You can test it by running pnpm start inside it. Multi-package live-reloading works if you do pnpm start --env live

First "base version" (in order to get it merged on main)

NOTE: This list is being triaged! Not everything listed here is going to be on the first "base version". Some stuff will be moved to their own issues and done later!

🛠️ Finish WIP components

• [x] 🔴 On the DMN Editor, implement "Documentation links" field for all nodes on Properties panel.
• [x] 🔴 On the DMN Editor, make it possible to edit a non-Struct Data Type's constraints.
• [x] On the DMN Editor, allow editing single-node styles / position / dimensions from the Properties panel. Make sure the Font selector displays a nice dropdown instead of a generic input text.
• [x] On the DMN Editor, make it possible to turn on/off custom node styles with an overlay flag.

Known crashes

• [x] Properties panel for Group nodes is crashing the Editor.
• [x] Add an Error Boundary to the DMN Editor component. When an error happens, have an option to "Undo last operation". Test with the crashes below. :D
• [x] Deleting TextAnnotation nodes is crashing the DMN Editor. Try with Associations going on both directions, one of them works, the other is broken.
• [x] Deleting an included model while there's external nodes on the diagram is crashing the DMN Editor
• [x] 🔴 On the DMN Editor, add a node from the Palette, press undo. Crash... We need to remove edges/nodes from selection arrays when they don't exist anymore.. not only when they're deleted. The DMN can change through props as well.
• [x] Copy/cut/paste for External nodes is crashing the DMN Editor. Ignore external nodes for now. Remember to remove them from references that other local nodes might have to them.
• [x] https://github.com/apache/incubator-kie-issues/issues/690

Structural bugs

• [x] Parameterize where the available models come from. VS Code, KIE Sandbox and Chrome Extension. What about the Standalone Editor. This is a prop on the DMN Editor itself --> includedModelsContextName and includedModelsContextDescription
• [x] Allow empty names for included models. Its elements can be used without having to prefix them with something..
• [x] The DMN Editor Data Type selector should have external Data Types too. Namespaced by the included model name.
• [x] On the DMN Editor's Boxed Expression Editor, renaming the top-level column header is not renaming the node that contains the open Boxed Expression
• [x] 🔴 The new DMN Editor cannot marshall Boxed Lists with empty expressions --> ERROR: Uncaught TypeError: Cannot read properties of null (reading '__$$element')
• [x] On the DMN Editor, creating a TextAnnotation from a Decision Service doesn't correctly create the Association edge.
• [x] 🔴 The DMN Editor has a weird repaint glich on Safari. (umd bundle of reactflow --> {handle:e,validHandleResult:t})=>h?"target)
• [x] 🔴 On the DMN Editor's Boxed Expression Editor, getDefaultExpressionDefinition needs actually create a very nice expression based on the typeRef of the containing expression... With widths that won't crop column names!
• [x] 🔴 DMN Editor state management bugs: (https://github.com/wbkd/react-flow/issues/2405).
• [x] 🔴 On the DMN Editor, snapping should be a non-destructive operation. onModelChange must not be called when changing the Snap grid.
• [x] On the DMN Editor, allow importing PMML models and make them work with BEE.
• [x] 🔴 Make sure everything that is pasted in any part of the DMN Editor has the IDs re-generated while keeping references on QNames and HREFs.
• [x] On the DMN Editor, disable the "Edit" button for external Decision/BKM nodes.
• [x] 🔴 On the DMN Editor, when a node is deleted, all references to it should be erased. Including @_hrefs, @_dmnElementRefs, and weaker refs like kie:ComponentWidthsExtension.
• [x] Boxed Expression widths set on the new DMN Editor don't work on the old DMN Editor. Explanation: The kie: namespace changed from http://www.drools.org/kie/dmn/1.2 to https://kie.org/dmn/extensions/1.0. Taking a file from the new Editor, changing the namespaced to DMN 1.2 and the kie: namespace to http://www.drools.org/kie/dmn/1.2 will make the widths work normally.
• [x] 🔴 ~On the DMN Editor, when an included model is deleted, all references to it must be erased. DMNShapes and requirements must be deleted.~ If a DMN has DMNShape/DMNEdges and @_href requirements pointing to unknown external elements, the Editor should continue working normally, without deleting them. Resoltuion: We're not deleting anything automatically. When an included model is deleted, a placeholder "unknown node" is shown. Users then will have the visual clue that something on their diagram doesn't look right.
• [x] 🔴 On the DMN Editor, changing the source handle of an edge is not working.
• [x] 🔴 On the DMN Editor, can't add new nodes on Firefox. Dragging nodes from the Palette doesn't work. See https://bugzilla.mozilla.org/show_bug.cgi?id=568313. Solution: Change the draggable Palette elements to divs instead of buttons.
• [x] 🔴 Make the DMN Editor convert all models to latest (1.5) (this means updating the xmlns attributes on the JSON). Version is hard-coded as 1.5. We need to actually convert the models to DMN 1.5. Remember to update the kie: namespace from http://www.drools.org/kie/dmn/1.2 to https://kie.org/dmn/extensions/1.0.

Decision Services

• [x] External Decision Service nodes should be collapsed by default.
• [x] ~On the DMN Editor, resizing a Decision Service must make the Divider Line push contained Decisions' positions to keep them inside their section.. (To make sure DRG doesn't changes when resizing a node!)~. Explanation: We're not going to do that. Instead, we're going to keep the old editor's approach. Resizing a DS won't move the Divider Line.
• [x] On the DMN Editor, it should be possible to move a Decision Service's Divider Line. The divider line will move until it hits either the lowest-positioned Output Decision, or the upper-most-positioned Encapsulated Decision.
• [x] On the DMN Editor, edges waypoints are not being updated for Decisions inside a Decision Service when the Decision Service is resized.

Edges

• [x] On the DMN Editor, selecting and dragging all nodes of Sample.dmn will lose edge magnet points on the top-level Decision.
• [x] 🔴 Safari and Firefox: On the DMN Editor, Edge Updaters are too big
• [x] 🔴 On the DMN Editor, Edge Updaters should be better positioned. They always need to be visible, independently of the handle the edge is connected to.
• [x] 🔴 On the DMN Editor, Cmd + click directly on Edge will keep adding it to the selected array, instead of toggling its selection.

Validations

• [x] 🔴 The DMN Editor must make sure all NamedElements inside the DMN have a locally unique name. Allowed characters for names should be validated as well.
• [x] 🔴 The DMN Editor must validate the names of included models. Allowed characters and uniqueness.
• [x] 🔴 The DMN Editor must validate that cycles do not exist on the DRG. ~Maybe d3-hierarchy helps?~. Solution: Reused the same mechanism we had for traversal and painting of dependencies/dependents of selected nodes. When a connection starts, a new derived state returns a set with all the dependencies of the source node. If a node is in this set, it's not a valid target of the connection, as it would create a cycle.

UX - Empty States

• [x] The DMN Editor should have a nice Empty State for the External Nodes popover.
• [x] The DMN Editor should have a nice Empty State for the Data Types when there's none.
• [x] The DMN Editor should present an Empty State for when includedModelsContextName doesn't have any available external models to be included.
• [x] The DMN Editor should show a nice message saying that "all models from includedModelsContextName were already included in 'This DMN'."
• [x] The DMN Editor should have a nice Empty State for the Diagram when there's no nodes.

UX - General

• [x] Cmd + A, then Esc is not de-selecting everything.
• [x] On the DMN Editor, Decision Service components (encapsulated/output Decisions, inputData/inputDecisions) should be nicely displayed on the Properties panel.
• [x] 🔴 Make it possible to rename included models. Remember to update all typeRefs with the new QName prefix!
• [x] 🔴 On the DMN Editor, allow Cut/Copy/Paste operations between Diagrams on different tabs. Remember to generate new IDs when pasting, as something copied once can be pasted multiple times!
• [x] The Data Types editor on the DMN Editor should have an option to paste top-level ItemDefinitions.
• [x] On the DMN Editor, each node type should have their own min width/height constraints.
• [x] The DMN Editor should have a button to generate a new ID/Namespace for the DMN file on the Global properties panel. This is important for when users copy/paste DMN files, ending up with IDs and Namespaces in duplicity.
• [x] On the DMN Editor's Boxed Expression Editor, the widths of nested Decision Tables are not correctly calculated when the Decision Table is nested inside another Boxed Expression.
• [x] On the DMN Editor, when copying Decisions and BKMs, the Widths of the contained Boxed Expressions should be copied too.
• [x] On the DMN Editor, when changing the name of an Included Model, PMML functions should be updated to have their "Document" field reflect the new name of the Included Model.
• [x] https://github.com/apache/incubator-kie-issues/issues/689

UI flaws

• [x] Improve node hierarchy highlight with proper styling and SVG stroke width etc. Edge markers are not being colored too.
• [x] 🔴 Knowledge Source node SVG is glitchy, specially when selected/hovered. The drop-shadow is wrong.

Other

• [x] Assess all FIXME: Tiago --> in packages/dmn-editor. Create issues/entries for each of them, grouping related entries. Can use this (https://github.com/apache/incubator-kie-issues/issues/625)
• [x] https://github.com/kiegroup/kie-issues/issues/4

---

Other information

THEORETICALLY 'INVALID' STATES THAT ARE TEMPORARILY ALLOWED IN THE EDITOR

• Boxed Expressions with empty expressions (Select expression)
• Floating external nodes... If they're not connected to any node on the Diagram, they're not part of the DRG.
• Decision Service nodes without any Output Decision.

INTENTIONALLY TEMPORARILY LEFT OUT

• Render DMNs with missing layout information (autolayout, ELK, mrtree)
• Guiding lines when repositioning/resizing elements on the diagram.
• Auto-target anywhere in the node for a fluid-auto-positioned edge, in additional to the already-existing 5 magnet points on nodes.
• ~Multiple DRDs. Only the first diagram listed on the DMN file will be used. Other diagrams are unchanged.~. EDIT: Decided to add it since it was "easy" enough :)

SPEC DISCREPANCIES

• Any type of Boxed Expression inside Boxed Relation cells

[tiagobento]

Just fixed the weird repaint glitch on Safari with the help of this

• https://stackoverflow.com/questions/39186554/understanding-safari-svg-redraw-bug-fix

Here's the commit, for anyone interested :)

• https://github.com/tiagobento/kie-tools/commit/31bab7132d056e8e5fd63ea0ba887b9a67c61d75

Before:

https://github.com/kiegroup/kie-issues/assets/1584568/c3e5bb94-d44e-477c-aed2-2973c0f40b03

After:

https://github.com/kiegroup/kie-issues/assets/1584568/bf5b41cf-5776-4dd9-ad04-0bff2f06e4ab

[tiagobento]

Btw the fix for the Safari repaint glitches is not perfect, but it makes the Editor usable :)

[tiagobento]

Copy/cut/paste working for nodes between different instances of the DMN Editor...

https://github.com/apache/incubator-kie-issues/assets/1584568/1be91d09-2796-4d86-ade8-4f3fb1cba1e6

[tiagobento]

Working on

"Make the DMN Editor convert all models to latest (1.5) (this means updating the xmlns attributes on the JSON). Version is hard-coded as 1.5. We need to actually convert the models to DMN 1.5. Remember to update the kie: namespace from http://www.drools.org/kie/dmn/1.2 to https://kie.org/dmn/extensions/1.0"

now... I'll group all the migrations that need to be done here:

1.0 → 1.1

• Namespaces
  • dmn: http://www.omg.org/spec/DMN/20130901 → http://www.omg.org/spec/DMN/20151101/dmn.xsd
• Root element from Definitions to definitions.
• typeRefs from tDMNElementReference to xsd:QName.

  1.1 → 1.2

• Namespaces
  • dmn: http://www.omg.org/spec/DMN/20151101/dmn.xsd → http://www.omg.org/spec/DMN/20180521/MODEL/
  • +dmndi: http://www.omg.org/spec/DMN/20180521/DMNDI/
  • +di: http://www.omg.org/spec/DMN/20180521/DI/
  • +dc: http://www.omg.org/spec/DMN/20180521/DC/
• FEEL namespace changed. Upgrade expressionLanguage and typeLanguage properties from http://www.omg.org/spec/FEEL/20140401 to http://www.omg.org/spec/DMN/20180521/FEEL/
• typeRefs from xsd:QName to xsd:string.

  1.2 → 1.3

• Namespaces
  • dmn: http://www.omg.org/spec/DMN/20180521/MODEL/ → https://www.omg.org/spec/DMN/20191111/MODEL/
  • dmndi: http://www.omg.org/spec/DMN/20180521/DMNDI/ → https://www.omg.org/spec/DMN/20191111/DMNDI/
  • di: No changes
  • dc: No changes
  • kie: http://www.drools.org/kie/dmn/1.2 to https://kie.org/dmn/extensions/1.0
• FEEL namespace changed. Upgrade expressionLanguage and typeLanguage properties from http://www.omg.org/spec/DMN/20180521/FEEL/ to https://www.omg.org/spec/DMN/20191111/FEEL/

  1.3 → 1.4

• Namespaces
  • dmn: https://www.omg.org/spec/DMN/20191111/MODEL/ → https://www.omg.org/spec/DMN/20211108/MODEL/
  • dmndi: No changes
  • di: No changes
  • dc: No changes
  • kie: No changes
• FEEL namespace changed. Upgrade expressionLanguage and typeLanguage properties from https://www.omg.org/spec/DMN/20191111/FEEL/ to https://www.omg.org/spec/DMN/20211108/FEEL/

  1.4 → 1.5

• Namespaces
  • dmn: https://www.omg.org/spec/DMN/20211108/MODEL/ → https://www.omg.org/spec/DMN/20230324/MODEL/
  • dmndi: https://www.omg.org/spec/DMN/20191111/DMNDI/ → https://www.omg.org/spec/DMN/20230324/DMNDI/
  • di: No changes
  • dc: No changes
  • kie: No changes
• FEEL namespace changed. Upgrade expressionLanguage and typeLanguage properties from https://www.omg.org/spec/DMN/20211108/FEEL/ to https://www.omg.org/spec/DMN/20230324/FEEL/
• Convert deprecated allowedValues to typeConstraint on ItemDefinitions. If the ItemDefinition is a collection, we need to "extract a new type" with the same "typeRef" as the ItemDefinition, add a description to it explaining why this was created (constraints are not projected to individual items on the list), and change the "typeRef" of the ItemDefinition to this new type. For non-collection types, simply copying allowedValues to typeConstraint is enough.