Graph Editor: Usability Improvements

[kwokcb]

This groups a set of suggestions / proposals for improving how users interact with the Graph Editor. The hope is to take the "best" ideas and designs and incorporate them if they will help interaction / usability.

Each item below is broken down into:

• Problem: description of issue
• Proposal: Proposal to address the issue
• Sub-issue: Link to sub-issue if applicable.

Folks can add / modify each item. Please up / down vote an item.

@lfl-eholthouser is the main stake-holder for this.

[kwokcb]

Problem

All UI is uniformly "grey" and aesthetically "dull"

Proposal

Add node color themes.

Issue

https://github.com/AcademySoftwareFoundation/MaterialX/issues/1373

[kwokcb]

Problem

It is very hard to distinguish data flow paths as they show up as overlapping lines with uniform appearance

Proposal

From Lee Kerley: Color graph edges to help to improve readability

[kwokcb]

Problem

It is very hard to find specific node types such as inputs, outputs, renderables, and conditions

Proposal

Add in color coding based on node-group or category. Note that this has overlap with color theming.

Issue

Note that node coloring is also mentioned by @dbsmythe in this grab-bag issue: https://github.com/AcademySoftwareFoundation/MaterialX/issues/1207

[kwokcb]

Problem

Text labels/ fields in the property editor and node graph are hard to distinguish from each other (e.g. for importance, for categorization etc)

Proposal

( Pulled from #1207 from @dbsmythe )

• I'd really like for nodes to indicate what their "nodecategory" is (e.g. add or over or fractal2d) right in the node itself, drawn in boldface at the top inside the node box, e.g. below the title with the node name and above the "out" output. Without this, it's hard to look at just the graph and see what mathematical operation it's doing.
• Similarly, in the Node Property Editor, it'd be nice if the nodecatagory value (e.g. the word "add" or "over" etc) was much more prominent: boldface, a different color, or something.

[kwokcb]

Problem

Visual clutter and space taken up for node ports on nodes

Proposal

Smarter Information hiding from @jstone-lucasfilm

• MaterialX supports additional UI features such as uifolder and uiadvanced, and some DCC tools have implemented these to good effect
• Some DCC tools bundle graph inputs into a single node, rather than distributing them throughout the graph, and this can potentially improve clarity.

Example from Houdini showing collapsible grouping (as well as pin and edge coloring by type)

[marwie]

Problem

Connected Inputs and Outputs don't show current (receiving or outgoing) values making it harder to debug the flow / state of a graph at a certain point ("does this input actually receive what I think it does")

Proposal

Display readonly values next to [type] (lower contrast, visually clear that they can't be edited)

[marwie]

Problem

When trying to resize the window sometimes the inner UI layout element gets selected (because it sets the arrow cursor on windows as well)

Proposal

Disable resizing behaviour of inner layout element (?)

https://github.com/AcademySoftwareFoundation/MaterialX/assets/5083203/1f1347d8-2b57-460c-963b-ff5cb32647c4

[marwie]

Problem

Refreshing a graph has to be done by re-opening the file. When debugging a graph that is generated by external tools I currently have to go to "File/Open File" and open the same graph again

Proposal

Add shortcut to reload the currently opened graph from disc (e.g. CTRL+R or F5)

[marwie]

Problem

Invalid graph information is not displayed in the graph and just in the console window

Proposal

Highlight nodes that are invalid / produce errors and display the error information in e.g. the Node Property Editor (?) as well

[marwie]

Problem

When adding a new node with keyboard (ENTER) the node list stays open and hides the newly created node which is confusing ("was the node created? where is it?")

Proposal

Hide the node list when selecting a node and accepting with keyboard (ENTER)

https://github.com/AcademySoftwareFoundation/MaterialX/assets/5083203/896431a1-809a-4b23-889c-a1248f795a39

[marwie]

Problem

Missing undo functionality

Proposal

Add undo/redo functionality

[marwie]

Problem

Already connected inputs have to first be disconnected to be then able to make a connection to another output

Proposal

Allow updating input connections by dragging another output connection over the port (replace the connection)

https://github.com/AcademySoftwareFoundation/MaterialX/assets/5083203/bb4ab8ed-50da-49bf-8de1-222101aae6d8

[marwie]

Problem

Connections dont start / end at the input/output dots but instead seem to use the center of the text. This is on windows 10. This is merely a visual problem :)

Proposal

Connections should start / end at the node edge or at the dot

[marwie]

Problem

Connections only work one-way / connections can not be made by dragging Input to Output ports

Please also see the comment below: https://github.com/AcademySoftwareFoundation/MaterialX/issues/1398#issuecomment-1684194636

Proposal

Allow connecting ports by dragging input over output port

https://github.com/AcademySoftwareFoundation/MaterialX/assets/5083203/7d690588-e5ab-44c2-b161-f21f95e2ea0d

[marwie]

Problem

Enabling Show All Inputs toggle pushes the toggle to the end of the list which forces scrolling to close / hide the list again

Proposed solution

Toggle stays at the same position and behaves more like a unfold header item

https://github.com/AcademySoftwareFoundation/MaterialX/assets/5083203/dcf3cf9f-697a-45e5-a952-48ba051d7f41

To illustrate:

https://github.com/AcademySoftwareFoundation/MaterialX/assets/5083203/0722ac79-b089-46ac-985f-63025a251908

Another possible idea would be to have a list popup which can be used to selectively add inputs into the UI (similar to the "Add nodes" popup list)

[marwie]

Problem

Material has to be saved to see the actual material code that is being written

Propsal

Add Viewer/Panel to display the MaterialX code that I would get when saving the graph (+1 if the code can highlight the part that I have currently selected in the visual graph + also visually mark/highlight the input nodes maybe)

[marwie]

Problem

CTRL+S always prompts with the filepath where the graph should be saved to (even if it was saved to a particular location already before)

Proposal

Remember last save location and override the file with CTRL+S as well as add CTRL+Shift+S for saving to a new location

Nice to have: hot reload (on material change on disc)

[marwie]

Problem

Graph Window doesn't show name of currently opened graph which becomes a problem when having multiple GraphView windows open ("I want to look at the marble_material")

Proposal

Show name of currently opened graph in the window (if any), e.g. marble_material - MaterialX Graph View title should start with the material name as it might be easily cut off otherwise, see screenshot below)

[marwie]

Problem

Selecting and cutting a group of nodes (CTRL+X) does sometimes only cut the connection and sometimes paste only the nodes BUT without the connection (loose the connections)

Solution

Connections should not be lost. When selecting nodes it should not just remove the connection. The nodes should be pasted in the same connected state as before

https://github.com/AcademySoftwareFoundation/MaterialX/assets/5083203/61b42c3e-85a1-4fed-bff4-b26cc9a99aba

[marwie]

Problem

editing swizzle channels is very very slow (everything freezes for 2-3 seconds) and I'm on a beefy PC

https://github.com/AcademySoftwareFoundation/MaterialX/assets/5083203/467b4324-2c3b-492e-b340-b1d30d1b7ecb

[marwie]

Problem

Exported graphs all have position values baked in which makes the graph harder to read when trying to understand / learn internals

Proposal

Add option to save without position / optional (meta-) data

[marwie]

Problem

On windows GraphView doesn't remember which graph I had opened before and where the window was on the screen

Proposal

Remember last opened material + position on screen when starting GraphView

[marwie]

Problem

Selecting a nodegraph node recompiles the material

Proposal

Dont recompile material when selecting a nodegraph node

https://github.com/AcademySoftwareFoundation/MaterialX/assets/5083203/aa55f7fe-b8ca-48ec-b038-f69636caea80

[marwie]

Problem

Open graph via file association is not supported

[marwie]

Problem

No way to remove inputs (e.g. if I have changed some values in the standard surface shader during testing)

Solution

Add a way to reset inputs to their default values which should also (perhaps) hide them again if "Show all inputs" is enabled.
Could be a context click option?

[marwie]

Problem

Property Editor UI is very cramped and leaves very little space between categories making it hard to read

Proposal

Add more space between items that don't immediately belong together For example between Category and Inputs headers should be ~half a line vertical space Similarly it would be helpful if the buttons below Inputs (namely "Show all inputs" and "Node Info") would be pushed down half a line. Header elements like "Category" and "Inputs" could be bold

Input elements could use the available horizontal space more properly:

Input element control items should be aligned for better readability

[marwie]

Problem

The console window doesn't print a message when i open a new graph. This makes it difficult to understand if validation errors in the console window where caused by the newly opened graph or were from a graph previously opened.

Proposal

Print a short message before opening a new graph to the console

<timestamp> Opening <filename> at <filepath>

[marwie]

Problem

Dragging a node does not select it. I was multiple times fooled by thinking I had the node selected and then e.g. delete the wrong node

I currently have to click the node to make a selection

Proposal

Dragging a node should select it (especially since it visually looks selected)

https://github.com/AcademySoftwareFoundation/MaterialX/assets/5083203/d91269b6-e389-4147-af64-b64669195ed0

Example from Unity ShaderGraph

https://github.com/AcademySoftwareFoundation/MaterialX/assets/5083203/89eef760-2619-4aef-bc46-c7dcd1c9c328

[marwie]

Problem

Property editor doesnt reflect multi-selection. It always just displays one of the selected nodes

Proposal

Property editor should render UI for all nodes selected.

https://github.com/AcademySoftwareFoundation/MaterialX/assets/5083203/21db509f-b903-4eef-b33f-614bb71ce82e

[marwie]

Problem

Copy paste of a node has unexpected inputs/defaults, pasted node is in a different state

Proposal

Copy paste should produce a copy of the node, the UI should look identical

https://github.com/AcademySoftwareFoundation/MaterialX/assets/5083203/eea0f386-ae01-422f-aa13-e5a46ce10a96

[marwie]

Problem

Automatic naming of nodes is confusing / misleading.

1) create a constant_vector2 node -> it is named constant_vector2 (also reflecting the type of the node, that's good) 2) create another constant_vector2 node for the second time -> it is named constant_vector3

Proposal

Node names ending with a number (reflecting their value type) should not just increment

• Perhaps the value type should not be included in the name but displayed differently on the node? Then it would become something like: constant_2 and vector3 be rendered elsewhere on the node (?)

• Slots could indicate the value type

  • e.g. value would become value (4) (see screenshot below from Unity)
  • alternatively value (xyzw) and value(rgba) which would indicate the type (might not work so well for matrix types tho)

https://github.com/AcademySoftwareFoundation/MaterialX/assets/5083203/77ba7620-d539-465b-b02d-3a3239720b3c

Example in Unity

[marwie]

Problem

Some input fields are very small and cutoff content

Proposal

Use the horizontal space available

somewhat related to the proposal mentioned above: https://github.com/AcademySoftwareFoundation/MaterialX/issues/1398#issuecomment-1683723538

[marwie]

Problem

Environment can not be changed which makes visual comparisons in some cases very hard / impossible

Proposal

Add option to change environment map (e.g. specify a EXR or HDR texture)

Standard material with a normal map, all other colors are white  

[marwie]

Problem

Scroll in PropertyEditor does also zoom in viewport in some cases Viewport interaction doesnt seem to check / play well with the scrollview

https://github.com/AcademySoftwareFoundation/MaterialX/assets/5083203/e3aab602-3fe6-44c8-86bb-9c5af8c1b613

[marwie]

Problem

Errors are hard to debug without visual indication / validation errors

Proposal

Visualize invalid connections and nodes

[marwie]

Problem

Trying to delete character in string field with DELETE key does delete the currently selected node

https://github.com/AcademySoftwareFoundation/MaterialX/assets/5083203/44fbc3e6-bcaf-4b73-a0c3-dc012511272f

[marwie]

Problem

When creating a new material I find myself always creating the same basic material

Proposal

new could have e.g. a standard_surface material already setup

or it could work like in Blender with a template material stored somewhere in the installation directory that I could also override and change to my liking.

I know (or think) it currently opens the marble material but I don't want to modify example materials for that purpose :)

[marwie]

Problem

Connections sometimes visually work both ways but are not actually working which is very confusing when debugging

Related to the comment above: https://github.com/AcademySoftwareFoundation/MaterialX/issues/1398#issuecomment-1680203806

the material from the video in case it's useful ```xml

```

https://github.com/AcademySoftwareFoundation/MaterialX/assets/5083203/28202f20-0e7b-4b01-a1a0-4869bd52ecc3

[marwie]

Request

Add file drop support to open a graph

https://github.com/AcademySoftwareFoundation/MaterialX/assets/5083203/0ba02e5a-d0fa-43e8-b202-fb7f08a031f1

[marwie]

Problem

Editor doesnt have a file icon (the same is true for the MaterialX Viewer)

[marwie]

Problem

Inputs list has unnecessary scrollbar

[marwie]

Problem

Alignment of output slots is not working correctly in certain cases

[marwie]

Problem

uiname is not used for node inputs (e.g. to name combine vector3 inputs x y z instead of in1 in2 in3)

[jstone-lucasfilm]

Thanks so much for this list, @marwie and @kwokcb, and I'm marking it as a Good First Issue for our upcoming Dev Days event in October, since developers can select a single issue from this list to tackle in a short time period.

[sandersteen]

Problem

Graph Window doesn't show name of currently opened graph which becomes a problem when having multiple GraphView windows open ("I want to look at the marble_material")

Proposal

Show name of currently opened graph in the window (if any), e.g. marble_material - MaterialX Graph View title should start with the material name as it might be easily cut off otherwise, see screenshot below)

Hey @jstone-lucasfilm , I had a quick look at this one to get myself more familiar with the document structure in MaterialX. I have a "fix" that calls glfwSetWindowTitle from the Graph::drawGraph function. I am not a big fan of this as it requires passing the GLFWwindow down from "main" to the above function call. I preferred it slightly over extracting the document out of the Graph class though, but both feel a bit mweh. Happy for better suggestions, or if this is the best we can do without doing bigger refactors I am happy to make a PR.

[jstone-lucasfilm]

Thanks for taking a look at this one, @sandersteen, and I'm CC'ing @lfl-eholthouser for her expertise on the systems that you're asking about in this question.

[lfl-eholthouser]

Hi @sandersteen! Thanks for looking at the issue. For the extracting the doc from the graph class option, would it be possible to just extract the name of the document instead of the whole document? Or are other aspects of the document needed?

[n-jay]

@jstone-lucasfilm I'm thinking of having a go at issue https://github.com/AcademySoftwareFoundation/MaterialX/issues/1398#issuecomment-1680360553. Hope that one is available.

[jstone-lucasfilm]

@n-jay Thanks for taking a look at this subtask, and I'm CC'ing @kwokcb, who did some closely related work in #1499.

I believe there are aspects of the subtask (e.g. the Save/SaveAs distinction) that didn't quite make their way into the final commit for #1499, but this earlier work should make a great starting point.

[n-jay]

Noted @jstone-lucasfilm, didn't notice that PR. Seems like I'd been running an outdated local main branch

but this earlier work should make a great starting point.

Yes, some of the doubts I had about the original implementation such as the _materialFilename had been addressed here, so it's cool to know I've been going down the correct path. 😄 I look forward to Bernard's response.