Improve UX for application properties

[dturanski]

Problem description:

The newest release of apps use a combination of properties, some are required, some are optional. More properties are exposed now, but we actually would be including even more. Take this stream definition for example:

s3 --spring.cloud.function.definition=s3Supplier,taskLaunchRequestFunction --file.consumer.mode=ref --s3.common.endpoint-url=http:<<lan-ip-address>:9000 --s3.common.path-style-access=true --s3.supplier.remote-dir=mybucket --s3.supplier.local-dir=/root/scdf/download --cloud.aws.credentials.accessKey=minio --cloud.aws.credentials.secretKey=minio123 --cloud.aws.region.static=us-east-1 --cloud.aws.stack.auto=false --task.launch.request.taskName=fileingest --task.launch.request.argExpressions='localFilePath=payload' | tasklauncher --spring.cloud.dataflow.client.server-uri=http://dataflow-server:9393

There are several categories of properties pertaining to the s3 source here:

• s3.supplier - Used by the base function used to implement the source. These are the core properties
• s3.common - Optional, shared by the s3Suppler, and s3Consumer
• cloud.aws - Used by spring-cloud-aws (currently not included in metadata)
• spring.cloud.function.definition - required for OOTB function composition (A new feature of all prepackaged sources makes several provided functions available for composition. This may be extended to other application types as well.).
  task
• task.launch.request - pertaining to the composed function. Each provided function has unique properties, currently not included in metadata.

Additionally, this, sftp source, ftp source support a persistent metadata store, we currently include metadata for several persistent metadata store implementations, not used in this stream definition (we may want to rethink this).

Presenting all these properties in a flattened form is problematic.

s3 property sheet 1:

s3 property sheet continued:

• Many properties which are irrelevant in this case are mixed with core properties.
• Useful properties related to composed functions are not included. Including them will make the presentation even more confusing.
• There are 2 region entries. One is the AWS S3 region, required for this source. The other is a Gemfire region name only relevant if the app is configured with the Gemfire Metadata Store.

Solution description:

We should prototype an improved UI for working with app properties. One proposal is to group the properties by category. So the user can understand the context of each group and expand each group as needed.

• There may be a primary or core group that lists properties at the top level, followed by optional sections. Available function properties and metadata store properties should always be treated as optional.

• There may be corresponding enhancements required to the metadata model required on the back end to support this

A good approach is to drive this from the UX we want, and make back end changes as necessary.

Description of alternatives:

Model after Preferences or Settings idioms. Show the most relavant/core properties with a way to select optional or additional properties, similar to the common Advanced... button.

[dturanski]

@oodamien - I discussed further with @markpollack Here's a very basic POC of the basic structure to start with, but should use something like an accordian widget, collapsible section for each group, etc. https://github.com/dturanski/grouped-properties-concept

[sabbyanandan]

Possibly also relates to https://github.com/spring-cloud/spring-cloud-dataflow-ui/issues/797.