ui-core: (feat) Switch views/apps definition to JSON

[ianchi]

This PR is a starting proposal on the format to use in order to switch to views/apps defined in JSON, as layed out in PRs #24 & #42

This is still WIP. The main ideas are in SCHEMA.md.

@jow- / @gmarcos87 / @nicopace Please send your views/comments/ideas, so we can better shape the best interface before starting to code an implementation.

Highlights

• The data schema defines the underlying data to be shown in the UI (uci configs, sections and options, and their types).
• The UI schema defines how this data is rendered as a form, e.g. the order of controls, their visibility, and the layout, and as status views.
• Additional information is embedded in this definitions to also include the data source. Initially only Ubus calls and UCI files are allowed

All this information is translated at runtime to generate the corresponding views and forms and bindings to the underling data.

Data Schema

All data schemes are defined using a subset of JsonSchema, with some additional custom properties. As the main source of data are

UCI config files

Each config file defines its structure with a schema with the same name, which describes possible sections types and the available options. The general structure is a wrapper object representing the config file, with properties representing each section type. Each section type should be an object whose properties are the available options.

If an UCI file doesn't have an associated schema file, a default one should be derived from all the current options present in the config.

See schema.data.md for detailed definition.

UI Schema

The UI schema describes the general layout of a view and is just a regular JSON object. It describes the view by means of different UI schema elements, which can be categorized into Navigation, Controls and Layouts. It also describes the bindings to the underling data (ubus/uci).

UI Schemas uses arrays to define properties, so as to define deterministic order of the UI elements.

When there is no explicit UI Schema defined, a default one should be generated based on the data schema.

Additional files

Additionally to the schemas, to complete the definition of a view the necessary ACLs and Menu definitions must be provided.

Example

There is an example schema file for UCI System in system.json

[ianchi]

Hi folks,

I haven't received any comments on this proposal. Anyway, I developed it a bit further and now there is a preliminary working version, implementing part of this ideas. You can see it in action with the system config, for which I wrote a schema.json file. (You have to select "network" in the menu and then "system" from the combo).

Ideas/comments/suggestions/acid critics are welcomed.

Some screenshots (no effort invested in css):

[ianchi]

Before polishing anything further, there are three main problems that need to be resolved if we aspire to have views defined solely in json files:

1. API to integrate ubus calls to provide values (specially list of valid items for combos), dependent on other options in the form
2. Simple way to define dependencies between fields of the config (so that they are showed only on some condition, for example)
3. So far I only made it work to show a single config file, in full. There should be a way to define a custom mix. (The UI Schema I outlined in my first post)

I'll start working on 1, it would be helpful to brainstorm ideas.

[ianchi]

Modified the documentation in SCHEMA.UCI.md to introduce logic to address the first two issues:

• Add definitions to specify binding of valid options to ubus calls or uci config
• Add definition to specify dependencies between options

I copy the relevant part, please comment:

---

Possible values

It is possible to restrict an option to a list of possible values. This can be done in two ways:

• Fixed value list, with standard enum field with array of valid values
• Dynamic values, with an extension to get the list from an Ubus service or UCI config, with enumBinding field
• Additionally, a non standard, additionalItems property can be set with a schema to signal that the list of possible values is not restrictive, and new values are allow providing that it validates against this schema.

If both enum and enumBinding are present, the union of both sets are valid values.

Static values

"enum": [value1, value2, value3]

Only the specified values are valid, they must be matching the type of the option.

Dynamic value list

Valid values are determined at run time as a result of an Ubus call or other UCI data.

"enumBinding": { 
  "ubus": {
    "service": "ubusServiceName",
    "method": "methodName",
    "params": {...},
    "listItem": "itemSelector",
    "valueItem": "itemSelector",
    "filterItem": "itemSelector",
    "filterPattern": "regular expression"
  },
  "uci": "uciSelector" | {
    "selector": "uciSelector",
    "filterPattern": "regular expression"
  }
}

Only one property (ubus/uci) can be present. If they both are, uci takes precedence.

The difference of getting data thru the uci property vs getting the same data thru ubus with a call to "uci get", is that when binding with uci you get a live binding to the data currently present on the form (even if it has not yet been saved), whereas the ubus call can only get what the router currently has saved.

Parsing result of UBUS call

To generate the list with valid values the listItem is extracted from the ubus response, which has to be an array proving the valid values, or an object whose properties represent the valid values. If no listItem is defined, the whole response is assumed as the list.

Each extracted value can be further refined and specific data extracted from it if a valueItem property is present.

The final value must always be of a primitive data type (string/number)

When the items must be composed from data of a UCI config file, a special

Filtering list items

The selected list can be optionally filtered applying a regexp to the data itself or to one of it items, the filterPattern, if it is an object/array

Bare in mind that regular expressions matching is not anchored, neither at the beginning nor at the end. This means, for instance, the pattern "es" matches "expression".

Selecting member

itemSelector allows to specify the object member/array item to use to extract the corresponding data. It traverses an object or array (or combination) to extract one specific single item.

The syntax is like normal object syntax in code:

• <propertyName>.<propertyName>.<propertyName>
• <propertyName>[<itemNumber>].<prpertyName>
• [<itemNumber>] (it's an exception when the returned data is directly an array instead of an object)

Parameter expansion

Prior to evaluating an itemSelector, matchExpr or filterRegExp a parameter expansion is performed, so that other UCI data from the same config file can be used to dynamically define them. Parameters are written in the syntax: ${uciSelector}

If the uciSelector returns an array (case of list options, or selection of multiple sections), it is expanded to only it's first item.

In any case that the referenced section/option is invalid, the parameter expands to an empty string.

Any number of parameters can be included in the expressions.

UCI Selectors

There are two kind of selectors: to retrieve the value of a specific option, to retrieve the name of a section

Returning option value or array of values (if it is a list option or multiple sections):

• (config.)(sectionName.)option to extract data from a fixed and known named section. Optionally a config file and section can be supplied, if either is omitted defaults to the current config and the current section. If "config" is supplied, "section" must also be supplied.

• (config.)sectionType[matchExpr].option to extract data from sections with the given type.

Returning sections' name:

• (config.)sectionType[matchExpr]

In either case [matchExpr] can be any of:

• []: empty expression select all sections
• [number]: select specific section by current ordinal position within the type
• [.name: 'pattern', option1:'pattern', option2: 'pattern'] Select sections that satisfies the condition between brackets. The criteria is defined by an option name without quotes followed by a colon, and then a singled quoted string with the regexp pattern to match.

Dependencies

Each option can specify dependencies on other options of the same section that must be met for the option to be available.

They can be defined with:

dependencies: { option1: boolean | [ "values" ] | "pattern" ,
  option2: boolean | [ "values" ] | "pattern" 
}

• option: boolean, if "true" that option must be present and non empty/false. If "false", it must be blank/
• option: ["values"], option must be set to any of the listed string values
• option: "pattern" option value must match the pattern

A shorthand to only validate presence of other options (like first alternative) is:

dependencies: ["options" ]

All dependencies must be met to validate.

No cyclic dependencies check is made, so be carefull no to make a loop of dependencies between options as it might create a situation where all options of the chain disappear, and there is no way to edit them.

[ianchi]

Changed enumBinding to use jsonPath notation, instead of a custom syntax. Even though it is not really a standard, it is much more known, and much more powerful. Besides we can have the same syntax for UCI selectors and for UBUS return selectors.

[ianchi]

Added basic binding to UCI selectors, using the same jsonPath notation.

[jow-]

I took a look at your work now and it looks impressive :) Unfortunately it will take me quite a while to get up to speed with type script and the new workflows required.

I skimmed over the changes but I am lost on to what to look for or how to test it. Managed to build ui-core and was able to launch it but not sure how to find a place using the enum bindings etc.

I suggest you that you simply merge this as baseline and we go on from there. I'll try to catch up as we go.

[jow-]

~To be more specific - how can I render a schema, e.g. the dropbear one?~ Nevermind, I switched the branch while the devlopment server was running and didn't notice that it redeployed the older code automatically.

[ianchi]

Hi @jow- Great to hear from you. I hope you were able to test. For now there are two dirty hooks into the menu. Selecting “network” brings up the page to edit any UCI config file. I made schemas only for “system” and “drop bear”, the rest are generated from the data.

Selecting the “system” menu, brings a page where you can try any ubus call. Usefull for testing a binding.

Now I wondered a bit to a sub project. When including “jsonPath “ it pulled in many dependencies and the app grew considerably. I like the simplicity and power of JsonPath notation for dealing with ubus responses, but don’t like its size. So I’m working on a new lib to handle that and also doing static eval of expressions to allow for simple transformations. I’ll be publishing the first version next week, and then integrate it back here.

[ianchi]

Use the new package ESpression to eval jsonPath expressions. It saves 50kb of gzipped space and will enable using mixed expressions when defining status output.