BUGFIX: Refactor node tree using Query DTOs

fixes: #54

TLDR; This PR replaces the mechanism through which Sitegeist.Archaeopteryx builds its own node tree with a custom-made backend API. The new approach has allowed to move a lot of responsibility from the client application to the server, which is much better suited to answer domain-specific questions. @nezaniel and I have designed this with Neos.Neos.Ui in mind. If the idea works for Archaeopteryx, we may be able to move the approach into the Neos UI as well.

General considerations

The entire approach is pretty much modeled after the ideas outlined in https://github.com/neos/neos-ui/pull/3331. However, the implementation in this PR is more Neos/Flow-idiomatic and requires less reflection. The overall idea is to move responsibility away from the client application to the server by creating a thin API layer, that maps directly to UI concepts.

This PR introduces multiple query data transfer objects (DTOs) with co-located query handlers. Each query has its own namespace beneath Application. Together they form the newly introduced Application layer.

*Query DTOs are final readonly classes with a fromArray static factory method. Each query DTO is accompanied by a *QueryHandler class, that is #[Flow\Scope('singleton')] and performs the actual operation. Each query DTO is also accompanied by a *QueryResult DTO class, that is also final readonly and additionally implements \JsonSerializable. The *QueryResult class is always the return type of the respective *QueryHandler->handle method.

There are some supporting DTO classes (like TreeNode or Breadcrumb). Those classes are usually co-located with the *QueryResult- or *Query DTO that are using them. In case multiple queries or query results need the same DTO, those DTOs have been moved into the Application\Shared namespace.

Each query has its own Controller. For this, a little bit of framework code has been introduced to reduce boilerplate. But in principle, the controllers do nothing more but to implement Flow's ControllerInterface and run their logic directly in processRequest (so: no actions or any other MVC concepts). All query controllers have very limited responsibility: Deserialize the query, invoke the query handler, serialize and return the query result. Everything else is responsibility of the query handler.

Also, all query controllers are folded into the Neos backend authentication. Neos.Neos:AbstractEditor is given access by default.

The Queries

Query `GetTree`

Screenshot_2024-05-15_16-26-01 Node Tree GetTree is used to render the node tree

Property Name	Type	Description
`workspaceName`	`string`	The name of the workspace the user is currently looking at.
`dimensionValues`	`array`	The exact combination of dimension values the user is currently looking at.
`startingPoint`	`NodePath`	The starting point for fetching the tree. (This is reflected by `startingPoint` in the editor configuration)
`loadingDepth`	`int`	The maximum traversal depth for fetching the tree. (This is reflected by `loadingDepth` in the editor configuration)
`baseNodeTypeFilter`	`string`	A filter string to match the type of every node in the tree against. Nodes with a type that does not match this filter never show up in the tree. (This is reflected by `baseNodeType` in the editor configuration)
`linkableNodeTypes`	`NodeTypeNames`	A list of node type names to mark all nodes that are allowed to be used as link targets. (This is reflected by `allowedNodeTypes` in the editor configuration)
`narrowNodeTypeFilter`	`string`	An additional node type filter selected by the user. If this is set, every node with a type that matches this filter will be retrieved and the tree will be build from the bottom-up.
`searchTerm`	`string`	A search term to find nodes by their property contents. If this is set, every node with a type that matches the search term will be retrieved and the tree will be build from the bottom-up.
`selectedNodeId` (optional)	`NodeAggregateIdentifier`	The identifier of the node that is currently selected. If this is set, the selected node (if it exists) will be guaranteed to be part of the resulting tree. If it resides deeper in the tree than `loadingDepth` would allow, it's entire branch will be build from the bottom-up

The GetTree query returns a hierarchy of TreeNode DTOs, which contain all the information the UI needs to render a tree. The tree is rendered with the node at startingPoint at its root.

By default, the depth of the rendered tree will be limited by loadingDepth. Descendants that are excluded thusly, can later be loaded using the GetChildrenForTreeNode query.

If searchTerm is set, the query will perform a search instead and return a tree containing every node that matches the searchTerm plus its ancestors (until startingPoint).

The query is informed by the editor configuration, by what is currently set in the Neos UI document tree (initially) and by user input.

If a node is selected whose depth in the tree exceeds loadingDepth and the Archaeopteryx dialog is opened after that, the tree will be loaded down to the depth of the selected node, including all ancestors and (ancestor-)siblings up to startingPoint.

EXAMPLE

Query:

https://my-neos-dev-instance.ddev.site/sitegeist/archaeopteryx/get-tree?workspaceName=user-admin&dimensionValues%5Blanguage%5D%5B%5D=en_US&dimensionValues%5BtargetGroup%5D%5B%5D=private&startingPoint=%2Fsites%2Fvendor-site&loadingDepth=1&baseNodeTypeFilter=Vendor.Site%3ADocument&narrowNodeTypeFilter=&searchTerm=&selectedNodeId=60c3de73-bb08-41fc-b73a-0c252cdf41d6

Result:

{
    "success": {
        "root": {
            "nodeAggregateIdentifier": "7d1a814b-f19b-4801-8621-eb8889fe638b",
            "icon": "globe",
            "label": "Example Site",
            "nodeTypeLabel": "Home Page",
            "isMatchedByFilter": true,
            "isLinkable": true,
            "isDisabled": false,
            "isHiddenInMenu": false,
            "hasScheduledDisabledState": false,
            "hasUnloadedChildren": false,
            "children": [
                {
                    "nodeAggregateIdentifier": "036b0e4c-e931-56b9-db53-7b4a9c5ded26",
                    "icon": "exclamation-triangle",
                    "label": "404"
                    // ...
                },
                {
                    "nodeAggregateIdentifier": "1e2948de-1e73-4688-9ae7-29a932e43eed",
                    "icon": "file",
                    "label": "About us"
                    // ...
                }
                // ...
            ]
        }
    }
}

`baseNodeTypeFilter` vs. `narrowNodeTypeFilter` vs. `allowedNodeTypes (???)`

I found it a bit confusing to differentiate between all those filters in relation to the tree.

There is a thing called baseNodeTypeFilter, which is configured via the editor configuration (analogous to how that's done with the document and content trees in Neos UI). The configuration key just says baseNodeType, but it is actually a filter string. This filter determines which node types can show up in the tree. So, any node type that doesn't match this filter will never show up in the tree, regardless of any other configuration or user selection.

Then there's a thing, that I called narrowNodeTypeFilter in this PR. This is the user-selected node type filter. If a user selects a filter option, the GetTreeQuery will actually perform a search and return all nodes that match the node type filter plus all of their ancestors.

There's also the configuration option allowedNodeTypes. This has nothing to do with filters at all. This option determines which node types are allowed to be linked (introduced in https://github.com/sitegeist/Sitegeist.Archaeopteryx/pull/11).

DTO `TreeNode`

Screenshot_2024-05-15_14-42-49 Tree Node States The various states a tree node can be in

Property Name	Type	Description
`nodeAggregateIdentifier`	NodeAggregateIdentifier	The ID of the (CR) Node aggregate that is represented by this tree node
`icon`	string	The icon for the tree node (by default taken from the node type's UI configuration)
`label`	string	The label for the tree node (by default this is just the (CR) node label)
`nodeTypeLabel`	string	The node type label (This is needed for the `title` attribute of the tree node icon)
`isMatchedByFilter`	bool	Flag that tells us whether this tree node matches the filter (`searchTerm` and/or `narrowNodeTypeFilter`) given in the query. If this is `false` the tree node will appear with reduced opacity
`isLinkable`	bool	Flag that tells us whether this tree node is allowed to be linked to (as per `allowedNodeTypes` configuration)
`isDisabled`	bool	Flag that tells us whether the node is disabled (or pre-9.0: "hidden"). If this is `true`, a little red icon will mark this tree node as disabled
`isHiddenInMenu`	bool	Flag that tells us whether the node is hidden in menus (or pre-9.0: "hidden in index"). If this is `true` the tree node will appear with reduced opacity
`hasScheduledDisabledState`	bool	Flag that tells us whether this node is scheduled to be disabled (determined by `hiddenBeforeDateTime` and `hiddenAfterDateTime`). If this is `true`, a little clock icon will mark this tree node as scheduled to be disabled
`hasUnloadedChildren`	bool	Flag that indicates whether this node has children that are not included in this result set because of `loadingDepth`
`children`	TreeNodes	The children of this tree node. If the depth of this node equals `loadingDepth`, this array will be empty and `hasUnloadedChildren` will be true

Query `GetChildrenForTreeNode`

https://github.com/sitegeist/Sitegeist.Archaeopteryx/assets/2522299/7372524e-d8c4-42b4-ae7f-ff5adf7e7f08

Loading children that have been excluded due to loadingDepth

Property Name	Type	Description
`workspaceName`	string	The name of the workspace the user is currently looking at.
`dimensionValues`	array	The exact combination of dimension values the user is currently looking at.
`treeNodeId`	NodeAggregateIdentifier	The ID of the (CR) node aggregate whose children shall be retrieved
`nodeTypeFilter`	string	A node type filter to match the children against. Children that do not match this filter will not be loaded.
`linkableNodeTypes`	NodeTypeNames	A list of node type names to mark all nodes that are allowed to be used as link targets. (This is reflected by `allowedNodeTypes` in the editor configuration)

This query loads a single level of tree nodes for the parent identified by treeNodeId. This is used to load children that have been initially excluded due to loadingDepth in the GetTree query. For plausible results, the workspaceName, dimensionValues and linkableNodeTypes properties should be the same as with the initial GetTree query.

EXAMPLE

Query:

https://my-neos-dev-instance.ddev.site/sitegeist/archaeopteryx/get-children-for-tree-node?workspaceName=user-admin&dimensionValues%5Blanguage%5D%5B%5D=en_US&dimensionValues%5BtargetGroup%5D%5B%5D=private&treeNodeId=1e2948de-1e73-4688-9ae7-29a932e43eed&nodeTypeFilter=Vendor.Site%3ADocument

Result:

{
    "success": {
        "children": [
            {
                "nodeAggregateIdentifier": "0ba7e951-2e2c-4447-9ac9-6f84995e219d",
                "icon": "map",
                "label": "Where to find us",
                "nodeTypeLabel": "Map",
                "isMatchedByFilter": true,
                "isLinkable": true,
                "isDisabled": false,
                "isHiddenInMenu": false,
                "hasScheduledDisabledState": false,
                "hasUnloadedChildren": true,
                "children": []
            },
            // ...
        ]
    }
}

Query `GetNodeSummary`

Screenshot_2024-05-15_14-41-53 NodeSummary

The node summary used for Archaeopteryx' main dialog

Property Name	Type	Description
`workspaceName`	string	The name of the workspace the user is currently looking at.
`dimensionValues`	array	The exact combination of dimension values the user is currently looking at.
`nodeId`	NodeAggregateIdentifier	The ID of the (CR) node aggregate that shall be summarized

When a link is selected, Archaeopteryx displays a panel in its main dialog that summarizes the link. The GetNodeSummary query informs this panel.

EXAMPLE

Query:

https://my-neos-dev-instance.ddev.site/sitegeist/archaeopteryx/get-node-summary?workspaceName=user-admin&dimensionValues%5Blanguage%5D%5B%5D=en_US&dimensionValues%5BtargetGroup%5D%5B%5D=private&nodeId=0ba7e951-2e2c-4447-9ac9-6f84995e219d

Result:

{
    "success": {
        "icon": "map",
        "label": "Where to find us",
        "uri": "node://0ba7e951-2e2c-4447-9ac9-6f84995e219d",
        "breadcrumbs": [
            {
                "icon": "globe",
                "label": "Example Site"
            },
            {
                "icon": "file",
                "label": "About us"
            },
            {
                "icon": "map",
                "label": "Where to find us"
            }
        ]
    }
}

Query `GetNodeTypeFilterOptions`

Screenshot_2024-05-15_15-02-16 Narrow Node Type Filter The node type filter options as displayed in the select box in Archaeopteryx' main dialog

Property Name	Type	Description
`baseNodeTypeFilter`	string	If no `nodeTypePreset`s are configured, the resulting options will consist of all non-abstract node types that match this filter string

The result of the GetNodeTypeFilterOptions query informs the node type filter select box in Archaeopteryx' main dialog.

Analogous to Neos.Neos.Ui, it will use nodeTreePresets to calculate the filter options, if any are configured. If not, the filter options will simply be a list of all non-abstract node types that match the given baseNodeTypeFilter.

EXAMPLE

Query:

https://my-neos-dev-instance.ddev.site/sitegeist/archaeopteryx/get-node-type-filter-options?baseNodeTypeFilter=Vendor.Site%3ADocument

Result:

{
  "success": {
    "options": [
      {
        "value": "Vendor.Site:Document,!Vendor.Site:Mixin.Type.Foo",
        "icon": "filter",
        "label": "Not the Foos"
      },
      {
        "value": "Vendor.Site:Document,!Vendor.Site:Mixin.Type.Bar",
        "icon": "filter",
        "label": "Not the Bars"
      }
    ]
  }
}

sitegeist / Sitegeist.Archaeopteryx