Document how to approach UX

[bgrant0607]

This is another issue filed to remind me to write a doc about how to think about UX, as a follow on to #3131.

I mentioned that separating the code from the data and exposing the resources that packages contain enables us to apply the full toolbox of standard UX principles, techniques, and tools to help package authors and consumers navigate, understand, and customize the package contents.

A question that often comes up is: how will users know what to change without parameters?

My primary thinking about that is how they know in any GUIs, terminal consoles, CLIs with interactive prompts, chatbots, etc.: step by step guidance for supported CUJs. We can also use progressive disclosure, hiding advanced features, and so on to mitigate domain complexity. My go-to example of this is TurboTax’s step-by-step mode, which asks questions and fills in details. You can break out of this and fill in data in any order. You can also switch from the high-level wizard to the low-level form view and change information without breaking the high-level wizard – they are interoperable views of the underlying data. Transformers and generators can do this if they assume they are not exclusively operating on the data. They should not clobber or clear fields they don’t control and should respect overrides of “default” values. In Kubernetes, ReplicaSet gets this right in that admission controllers, scheduler, vertical pod autoscaler, kubelet, and other actors in the system can change pods. The Deployment controller gets it wrong and it doesn’t behave well if the ReplicaSets it creates are modified in any way. (Also, no default values should be set in pod templates in any controllers – that’s a bug.) Live-state UIs and CLIs are generally interoperable.

In #3121, I also mentioned that if we specify which functions transform upstream resources into the downstream ones (at least for concrete deployment packages) and their input types are well defined, then we could generate or prompt for those inputs at the appropriate times.

Abstraction design for generators is a topic unto itself and deserves a separate issue. Unsurprisingly, when working on the namespace provisioning example, we found a low-value abstraction just got in our way, and we eliminated it.

Other thoughts:

Understanding:

• Generating docs is useful. We have a function that does that. Some of that info we could surface in the UI. Or show the readme.
• I've suggested we could leverage kubectl explain (#2331), or do something similar, in order to get resource information.
• We could generate a resource graph diagram for inter-dependent resources
• Something like kubectl get and/or describe could show summaries of the current values in the blueprint. Right now we just show name and namespace in the list view and all attributes in the resource view.

Navigating:

• We have a flat resource list in the UI. We could display something more similar to kpt pkg tree.
• We could identify "interesting" attributes, categorized by type of concern. I imagine these working similarly to value transformers (I'll file another issue about those).
• We could provide more navigation structure
• More documentation and help

Customizing:

• automate, automate, automate
• context injection (e.g., identity, context / config)
• data import
• best-practice defaults, suggested values
• convention over configuration
• In the prototype UI (pre-Backstage), we flagged placeholder values containing the string "example". Simple, but it worked pretty well.
• We have some function discovery (#2333), but we need to be able to filter by resource type.
• autocompletion could be used almost like a menu system
• suggest subsequent operations based on prior operations
• Each of the set-foo functions could have corresponding get-foo functions that identify the locations to change and their current values.
• As with the current setter tables in the documentation, we could summarize the number of replacements for each function. Or, we showed diffs when proposing changes in the prototype UI. We could show diffs in more cases, such as before/after kpt fn render, diff with upstream, etc.
• Function idea: we could generate required RBAC for the resources in a package
• sanity checks: schema validation, value validation, referential integrity, ensure that all placeholders have been replaced, linter, policy enforcement

[bgrant0607]

Another idea discussed: allow lists and deny lists of attributes that could be changed in deployments. These could also be used to direct users towards changeable attributes, if significantly smaller than the package as a whole.

[bgrant0607]

A related question from slack: "is there a way to provide hints to a UI or CLI tool, what fields & resources a user could be interested in modifying? I understand a kptfile can refer to local configmap yamls. But I'd envision a case where 1 deployment 'blueprint/template' package has 4 blueprints cloned as subpackages. the deployment blueprint when turned into a real deployable package will have so much yaml (resources+fields) that 90% of the users won't be interested in, it would be nice to be able to easily put a 'view' on top, which can be used to guide the user to resources and fields the author has provided great defaults for, but understands that a large group of users would still like to modify. By standardizing on some kind of persona-hints data structure, CLI (3rd party) tooling could provide the same great experience as the UI"

We do want to simplify configuration without abstraction, which often erodes (#3288, #3131).

Except for application-specific configuration, very often configuration patterns for particular resource types are similar across packages. In particular, attributes that must be specialized to the use case are often the same. Thus, user interface surfaces, such as GUIs, and KRM functions targeting particular resource types (e.g. value transformers #3155) should be more reusable than typical configuration generators.

Some possible UX techniques are discussed above. While it may be possible to create some kind of markup or other package-specific data-driven value prompters, setters backfired so terribly that I'd be reluctant to support that until other approaches were exhausted first.

[bgrant0607]

Some quantitative UX productivity measures:

• Number of fine-grained steps required for a task: commands, clicks, IDE searches, lines of config written by hand, etc.
• Number of lines read or changed
• Number of context switches (e.g., to look up something in docs or help, or switches between tools)
• Number of decision points
• Number of times stumped during a task
• Number of different tools required to complete a task
• Number of errors encountered / mistakes made
• Task completion latency
• Task completion success rate

[bgrant0607]

Here's an example article with strategies for coping with complex domains: https://www.nngroup.com/articles/complex-application-design/

[bgrant0607]

Application configuration should provide a rich example for testing out some of these approaches: #3351