Database editor user interface development

[nelliputkonen]

This is a parent issue for all DB editor UI development issues. The parent issue lists all current functionalities (bullets) in DB editor (Toolbox 0.8) and lists suggested improvements (checkboxes) under functionalities. The suggestions can be easily opened as new issues hierarchically under the parent issue. The list is not yet comprehensive and the suggested improvements as well as their order of priority has to be discussed (e.g. starting in Leuven sessions Jan 2024). The listing also needs cross-checking with existing issues and planned improvements - therefore this is a live listing that should be edited as the UI development process moves along.

Spine Toolbox database editor

User function categories:

1. Create a modelling tool specific database format (Populate database with modelling tool specific entity classes and parameter definitions)
2. Create a model specific database with data (Populate database with model specific entities and parameters)
3. Edit data and create scenarios based on alternative parameter values
4. Check model run results for checking if model works
5. ~Analyze full set of results~

General UX suggestions:

• [ ] #2588
• [ ] When opening for the first time, there could be a tour to show important locations. This could be lated accessed through Help menu in the top menu (or hamburger menu if that stays)
• [ ] Analyse how to organize the docs (Entity tree, Parameter value etc.) in the most intuitive manner, and which are active from beginning and which can be accessed via menus. Use benchmarks of software people are used to, that work well and create options and make user testing.
• [ ] Allow creation of use modes on multiple levels
  1. Main user function: Input data creator mode / Input data user mode / Result checking mode / ~Result analysis mode~ - this could be a selection for the user or input data vs. results could also be detected automatically? Each mode would have different default view while all features are still accessible. The modes need careful planning of their own if this suggestion is put forward.
  2. Customized entity tree: In the creator mode you can customize the views shown in user mode (selection of views available for the user). This would include custom order of entity classes, hierachy, preconditions and highlighting.
  3. Customized entity selections within entity class. For example, in Flextool you could have a node uses: energy_balance / commodity_market / storage. As the user creates a new node and selects its use, empty rows for necessary parameters appear to parameter value list

Top menu and Toolbar

• [x] Add frequently used menu entries to toolbar
  • Commit (with message field),
  • Purge and
  • Graph/Table view toggle
  • Switch between databases in editor tabs
  • Open a new database by '+' next to tabs
  • Toolbar
    • Backward
      • [x] This is undo. Consider changing the icon to curved arrow
    • Forward
      • [x] This is moving forward in the change stack (redo undo). Consider changing the icon to curved arrow
    • Reload
    • Open database in project
      • [x] Currently in "+" sign on top, in the database icon on toolbar, and under file menu, are all three necessary? Remove this one?
    • Path of open database
    • Filter
      • Scenario filter only (scenario filtering does not work yet from the scenario widget --> will work and then this will be removed)
    • Toolbox logo
      • [x] ~Switches to Toolbox project GUI, should it rather open a webpage?~ Kept as is, added github item to help -menu
  • Customize windows by opening trees in new windows, closing them or reorganizing

Hamburger menu (in Toolbar)

• [x] #2774
• File
  • New database
  • Open database
  • Add database
  • Import data
  • Export data
  • Export session
• Edit
  • Undo
  • Redo
  • Copy names as text
  • Paste
  • Purge
    • [x] Needed quite often, bring to toolbar?
  • Vacuum
• View
  • Table
  • Graph
  • Pivot table
    • Value
    • Index
    • Relationships
    • Scenarios
      • [x] Bring icons to Toolbar that can toggle visibility of table, graph and pivot widgets
      • [x] Create a reset icon for the views to the Toolbar
  • Docks
    • [x] By clicking Table/Graph you can restore default, there should be an option to "Restore default layout" (and maybe clicking table/graph shouldn't reset)
    • Entity tree (default: visible)
    • Parameter value (default: visible)
    • Parameter definition (default: visible, as second tab of Parameter value)
    • Entity alternative (default: visible, as third tab of Parameter value)
    • Pivot table (default: not visible)
    • Frozen table (default: not visible)
    • Entity graph (default: visible only if Graph view is selected)
    • Parameter value list (default: visible)
    • Alternative (default: visible)
      • [ ] Rename to Alternative tree to align with other docs?
    • Scenario tree (default: visible)
    • Metadata (default: visible, as second tab of Parameter value list)
    • Item metadata (default: visible, as third tab of Parameter value list)
    • Exports (default: not visible)
  • Session
    • Commit
      • [x] Bring Commit to toolbar
    • Rollback
    • History
  • User guide
    • [x] Typically would have this under menu item 'Help'
  • Settings
    • [x] Typically would have this under menu item 'File' or 'Edit'
  • Close

Entity tree (used for creating/editing entity classes and entities)

• [x] #2773
• [ ] Add an icon or list menu item to describe what entity classes and entities are and link to tutorial in github
• List of all entity classes
  • Click to filter parameter values of clicked class
  • Arrow to collapse entities under entity class
    • Click to filter parameter values of clicked entity
    • Arrow to collapse relationships
      • Click to filter parameter values of clicked relationship
• Entity class right-click menu
  • [ ] Currently unavailable menu selections are greyed out - they could be simply hidden (at least those ones that don't concern the entity you have right-clicked)
  • Copy name(s) as text
  • Add entity classes
    • [ ] Entity class features would not need to be available all the time, you could somehow select whether you are building classes or simply using pre-defined classes, or move these options e.g. under selection in the View menu. Optionally you could have two dropdown menus next to File, Edit... called 'Structural building' and 'Model building' and add most important options from right-click menus there.
    • [ ] Allow organizing entities in several ways: a) group under titles (e.g. model and temporal settings -> all related entities), b) Organize entities in the order the user is expected to use them (e.g. model and temporal settings, node, unit, commodity, profile etc), c) Toggle entity visibility based on other entities (e.g. the relationships of an entity appear only if entity exists, or upDown entity appears if reserves are defined) d) Highlight mandatory entities (in order to do meaningful modelling)
  • Add entities
    • [x] #2788
  • Manage elements
  • Manage members
  • Select superclass
  • Find next occurence
  • Edit
    • [ ] Show a keyboard shortcut for editing the entity (in the right-click menu)
  • Remove
    • [ ] Show a keyboard shortcut for removing the entity (in the right-click menu)
  • Duplicate entity
    • [ ] Show a keyboard shortcut to duplicate the entity (in the right-click menu)
  • Export
    • [ ] export dialog could be expanded to have similar capabilities as the exporter tool (but for single shot exports)
  • Fully expand
  • Fully collapse
  • Hide empty classes (toggle)
    • [ ] This is useful and could be moved e.g. on top of the entity tree as toggle button between Basic classes (only mandatory classes + classes that have entities) / All available classes
    • [ ] But needs to be thought together with 'display modes' that could better help to filter needed classes

Parameter value (used for assigning parameters and parameter values to entities)

• [ ] Add an icon or list menu item to describe what parameters and parameter values are and link to tutorial in github
• entity_class_name (1st column)
  • click header row (1st row) to filter
    • [ ] Add filter icons next to names for clarity (same for all 1st row)
  • click last (empty) row for dropdown menu of all entity classes
    • [ ] Could only show entity classes with entities
  • order by
• entity_byname (2nd column)
  • click name (1st row) to filter
  • dropdown menu of all entities available for the entity_class
    • start typing to filter entities
• parameter_name
  • click name (1st row) to filter
  • dropdown menu of all available parameters for the entity class
    • [ ] #2793
    • [ ] #2710
    • [ ] Right-click menu could have item view Parameter definition (especially if Parameter definition table is not visible by default like suggested)
  • order by
• alternative_name
  • [ ] Should the order of alternative_name and value column be changed for clarity? (Nelli thinks so, Juha doesn't)
  • [ ] Give each parameter a default alternative (the last one used)
  • order by
• value
  • [x] #2791
  • [ ] Highlight if data is missing or there are errors in data
  • [ ] For future 'limited parameters' (that have a selection of sub-parameters available to be filled, defined in parameter_definition), enable a view mode that lists them in separate rows
  • [ ] For timeseries type of parameter values, consider allowing link to csv (this is a Plexos functionality)
• database
  • [ ] Toggle parameters by database? This way could get rid of repeating the same database name
  • order by
• Right-click menu (available at any place in table, same in Parameter value, Parameter definition and Entity alternative)
  • [ ] Currently unavailable menu selections are greyed out - they could be simply hidden (Juha: I think it's better to keep them greyed - indicates if you are doing something wrong) (Nelli: agree)
  • Edit
  • Plot
  • Copy name(s) as text
  • Paste
  • Remove row(s)
  • Filter by
  • Filter excluding
  • Clear all filters

Parameter definition (used for viewing the list of all parameters or adding/editing parameter definitions)

• [ ] Add an icon or list menu item to describe what parameter definitions ad are and link to tutorial in github
• [ ] Consider if this Dock needs to be visible by default. Could access definition by right-clicking parameter for viewing purposes.
• [ ] Juha: Maybe just depends on the Mode, also for the next one.
• [ ] Parameter class adding features would not need to be available all the time, you could select whether you are building classes or simply using pre-defined classes, or move these options e.g. under selection in the View menu
• [ ] Adding 'limited parameters', i.e. that have a selection of sub-parameters available to be filled, e.g. invest_parameters with a table that includes sub-parameters for lifetime, invest_cost and interest_rate Same table structure with filtering functionalities in header rows and clicking any row shows all possible selections
• entity_class_name (1st column)
• parameter_name (2nd column)
  • Add new parameter definition by inputting new parameter name to the last row
    • [ ] This can be hard to find. Add option to right click menu in Parameter definition table: Add new parameter definition
    • [ ] Also considering moving 'add new' to the first row - it could also function as a search/filter bar
• value_list_name (3rd column)
  • [ ] Parameter value list could open in new window if this is selected
• default value (4th column)
• description (5th column)
• database (6th column)
  • [ ] Toggle tree by database? This way could get rid of repeating the same database name
• [ ] Add here columns/options for parameter hierarchy, dependency on other parameters, mandatory/not (could also have more categories like common/advanced), allowed value types and validity checks

Entity alternative (used for selecting which entities are active in which alternative)

• [ ] Consider moving to the parameter_value table (as the first item for each entity) - this could also be a view mode option Same table structure with filtering functionalities in header rows and clicking any row shows all possible selections
• entity_class_name (1st column)
• entity_byname (2nd column)
• alternative_name (3rd column)
• active (4th column)
  • false/true
• database
  • [ ] Toggle tree by database? This way could get rid of repeating the same database name

Alternative (used to give alternative parameters/parameter values for entities for scenario creation)

• [ ] Add an icon or list menu item to describe what alterenatives are and link to tutorial in github
  • [ ] Rename to Alternative tree to align with other docs?
• List of alternatives in database
  • [ ] If no alternatives exist yet, add default alternative e.g. called "base"
  • Filter parameter_view by clicking alternative name
    • [ ] Add filtering also for graph_view
  • Add new by clicking last in list "Type new alternative name here..."

Scenario tree (used to build scenarios based on alternatives)

• [ ] Add an icon or list menu item to describe what scenarios are and link to tutorial in github
• List of scenarios in database
  • [ ] If no scenarios exist yet, add default scenario e.g. called "base" that includes the alternative called "base"
  • Filter by clicking scenario name
  • Arrow to collapse alternatives in scenario
    • Add new alternative by clicking last in list "Type new scenario alternative name here..."
  • Add new scenario by clicking last in list "Type new scenario name here..."

Parameter value list (used for defining a list of possible values for a parameter)

• [ ] Link Parameter definition (adding/editing) and Parameter value list together, because they are linked together (e.g. on tabs of the same Dock or Parameter value list as a sub-option of Parameter definition, a window that opens if you choose value_list_name as 'yes'). Parameter class adding features would not need to be available all the time, you could somehow select whether you are building classes or simply using pre-defined classes, or move these options e.g. under selection in the View menu
• List of all parameter value lists
  • Arrow to collapse possible values under parameter value list
  • Add new by clicking last in list "Enter new list value here..."
• Add new by clicking last in list "Type new list name here..."

Entity graph (used for visualizing entities and their relationships)

• Shows all filtered entity display icons with their relationships to other entities
  • [ ] Improve not clustering icons by adding GIS option or automatically spacing icons
  • [ ] Allow saving user-defined Entity graphs - or is it already allowed?
  • Right-click menu
    • [ ] Break down or cluster into shorter menu selection
    • Add entities
    • Search
    • Hide classes
    • Show
    • Show all
    • Prune classes
    • Restore
    • Restore all
    • Zoom
    • Arc length
    • Rotate
    • Auto-expand entities
    • Merge databases
    • Snap entities to grid
    • Max entity dimension count
    • Number of build interactions
    • Minimum distance between nodes (%)
    • Decay rate of attraction with distance
    • Select graph parameters
    • Select background image
    • Save positions
    • Clear saved positions
    • Save state
    • Load state
    • Remove state
    • Export as image
    • Expost as video
    • Rebuild

Not covered in this list (yet)

• Settings (under Hamburger menu)
• Pivot tables
• Frozen table
• Metadata
• Item metadata
• Exports

[OleMussmann]

Impressive what you put together in so little time @nelliputkonen . I only have time for a brief review right now. Below are my initial thoughts. I hope I find the time to go through your points again with the user interface in front of me later. When I quote your bullet points, they are in bold.

• What are User function categories? A workflow you want to analyze/showcase?
• Allow creation of use modes on multiple levels If "guessed", these need to be spot on, or very easy to change
• We need to carefully think about accessing functionality from multiple locations. It's a more fundamental question about design philosophy. The two extremes are 1) there should be only one, obvious way to do things; 2) Make functionality available in every location where it could be used from. I'm not saying we should pick one of those, but we should try to formulate a design approach and implement it consistently through the user interface.
• Move the hamburger menu to the top as dropdown menus (like in workflow editor and typical Windows programs) If I recall correctly, it's the "ribbon" style of user interface.
• Entity tree (used for creating/editing entity classes and entities) The concept of "entities" should probably be outlined in a tutorial that you mention. New users might not know what those mean yet. Same with "alternative names" and other concepts.
• Hide database column (if only viewing a single database) (then can also hide 1st columns name 'name') Should we really hide the 1st column? It would create two different views, depending on how many databases are shown. Could the name column also be used to re-name the database? Then we should definitely leave it.
• User interface changes should be done in dialogue with different types of users. What could be a quality-of-life improvement for someone, could brake the workflow from someone else.
• Entity graph (used for visualizing entities and their relationships) -> Right-click menu: This list is very long, could we break it down and cluster some of those in some sub-menus?

[nelliputkonen]

Thank you for the excellent comments @OleMussmann! My answers and comments in bold.

• What are User function categories? A workflow you want to analyze/showcase? The DB editor serves many functions, and these are the different use cases what the user needs the database editor for. Typically user is only using one of these at a time, or same user might not ever need all of them.

• Allow creation of use modes on multiple levels If "guessed", these need to be spot on, or very easy to change Indeed. I would go for very easy to change.

• We need to carefully think about accessing functionality from multiple locations. It's a more fundamental question about design philosophy. The two extremes are 1) there should be only one, obvious way to do things; 2) Make functionality available in every location where it could be used from. I'm not saying we should pick one of those, but we should try to formulate a design approach and implement it consistently through the user interface. Good point and new to me. Deserves a discussion.

• Move the hamburger menu to the top as dropdown menus (like in workflow editor and typical Windows programs) If I recall correctly, it's the "ribbon" style of user interface. Updated to listing

• Entity tree (used for creating/editing entity classes and entities) The concept of "entities" should probably be outlined in a tutorial that you mention. New users might not know what those mean yet. Same with "alternative names" and other concepts. Updated to listing an icon or menu selection to offer short description and link to tutorial in github

• Hide database column (if only viewing a single database) (then can also hide 1st columns name 'name') Should we really hide the 1st column? It would create two different views, depending on how many databases are shown. Could the name column also be used to re-name the database? Then we should definitely leave it. Updated to that maybe we toggle entity tree/parameters by database instead. This way we don't have to repeat the database name as typical use case (I guess) is a single database

• User interface changes should be done in dialogue with different types of users. What could be a quality-of-life improvement for someone, could brake the workflow from someone else. Yes, you are better experts on this and we need to discuss how to implement this

• Entity graph (used for visualizing entities and their relationships) -> Right-click menu: This list is very long, could we break it down and cluster some of those in some sub-menus? Updated to listing