Category symbols and general expression grammar

[nickfargo]

Taken from discussions in #15, #16

Background

StateExpressions, and States (via their StateMetaobject), are defined largely as collections of properties, like a plain object, but with the added structure of separating everything into distinct namespaces or categories, which include: (data methods events guards substates transitions).

State expressions would benefit from the ability to use simple stateless primitives (i.e. strings) to define indirections that reference other parts of itself, and to express simple common imperatives (e.g. executing a transition). Currently, support for this is limited to a few ad-hoc and disparate cases.

This situation could be improved by clearly specifying a set of category symbols (or “category sigils”) that could be included in state expression keys and selector strings, for use with accessors (object.state('...')), state expression values, and elsewhere — all of which would be defined in terms of a simple, explicit grammar.

Example 1

Under the rules currently in place for the StateExpression interpreter, how should one expect this string literal to be interpreted?

expression = state({
  admit: 'path'
});

The string path is intended here as a selector that filters for valid origin states … but that’s not clear; it must be inferred or drawn from implicit prior knowledge that admit is a guard action, and that the library will interpret an associated string value in this context as a state selector.

Example 2

Under different rules, how might one expect this string literal to be interpreted?

expression = state({
  admit: 'methodName'
});

Here the string methodName intends to refer to a function, in the role of a method, that would act as the declared guard’s predicate. But this isn’t clear either, for the same reasons in the previous example.

Whether a string value in an expression is meant to reference a state, a method, or what exactly, is not clear until the reader has studied and gained a sufficient knowledge of the API. Interpretation rules are dependent on a definition of context that isn’t particularly clear, and these hidden semantics make reasoning less easy than it could be.

Simple “sigils”

It would be helpful, therefore, to have a means by which one may disambiguate the categorization explicitly, rather than by inference or ad-hoc definitions. To that end, this proposal defines a simple set of special symbols, which, as part of a selector or string value inside a state expression, each represent a specific category:

• .: substate — Works mostly as per status quo; i.e. path.to.descendant.state.
• !: transition — This defines the separation between a State S on the state tree and either: (1) a TransitionExpression defined on (or resolvable by) S, or (2) an active Transition instance presently attached to S in the course of the transition’s traversal.
• ?: data — Properties stored in a State’s data. Deeply nested object properties are accessible after the ? with . in the usual fashion; e.g. state?long.key.path.
• ::: method
• :: event
• #: guard
• ~: channel — (Reserved; not implemented yet)
• @: history — (Reserved; not implemented yet)

With these terms, the previous examples disambiguate to

expression1 = state({
  '#admit': 'path' // a state
});

expression2 = state({
  '#admit': '::methodName'
});

General expression grammar

empty               ::= ''

data-symbol         ::= '?'
method-symbol       ::= '::'
event-symbol        ::= ':'
guard-symbol        ::= '#'
member-symbol       ::= '.'
transition-symbol   ::= '!'
channel-symbol      ::= '~'
history-symbol      ::= '@'

wildcard-symbol     ::= any-substate
                      | any-descendant
                      | any-state

any-substate        ::= '*'
any-descendant      ::= '**'
any-state           ::= '***'

name                ::= /^[$_A-Za-z][$\w]+$/
super-relation      ::= /^\.+$/

member-wildcard     ::= member-symbol wildcard-symbol

member              ::= member-symbol name

nested-member       ::= member
                      | member nested-member

data-selector       ::= data-symbol name nested-member?

method-selector     ::= method-symbol name

event-selector      ::= event-symbol name

guard-selector      ::= guard-symbol name

substate-selector   ::= member
                      | member substate-selector

transition-selector ::= transition-symbol empty
                      | transition-symbol name

path                ::= static-path
                      | transitional-path

static-path         ::= relative-path
                      | absolute-path

transitional-path   ::= static-path transition-selector

absolute-path       ::= empty
                      | wildcard-symbol
                      | name substate-selector? member-wildcard?

relative-path       ::= super-relation absolute-path

component-path      ::= path data-selector
                      | path method-selector
                      | path event-selector
                      | path guard-selector

Example 3: selector expressions

1: 'connectivity.online.ready!polling::doSomething'
2: 'connectivity.online?path.to.data.property'
3: 'connectivity.online:exit'
4: 'connectivity.online!'
5: '#enter'
6: ':enter'

1. A transition-symbol ! may precede the method-symbol ::. Depending on the context in which it is used, this string specifies a method named doSomething on either: (a) an active transition polling attached to state ready; or (b) a transition expression polling defined on or heritable by state ready.

2. A data-symbol ? may precede the member-symbol ., where it acts as the long key path separator for lookups of deeply nested properties. This string would have a meaning similar to the imperative expression

   object.state('connectivity.online').get('path.to.data.property');

3. The exit event defined on state online.
4. An anonymous transition, currently attached to state online.
5. A guard expression, with no state membership defined.
6. An event expression, with no state membership defined.

[zoomclub]

The "StateMetaObject" and it being a simple collection of properties rings well with me. Indeed, a configuration that has something in common with a "StateMetaObject" but also extends the concept by a stretch is at the core of each state in the FSM's I'm defining. These configurations are swappable, so that a given state can be loaded with alternate configurations. These configurations are pretty well another "softwired"JSON based layer of state within a state (like jam in a cookie). I'm not sure if you are considering also making a complete FSM into a JSON based configuration as well?

After much thought and many eureka moments over much time, I came to the realization that the configurations used by most of the states could be composed of separate categories of key-value properties, some being non-terminal and others terminal. These categories are somewhat similar to the "Sigils" mentioned above. However, there is a difference to note between the more hardwired JS coded states of the FSM and the softwired JSON configuration a given state uses in my app.

With my configuration being JSON these categories are just sections defined as respective chucks of Object Notation, using the property map and array features of JSON to build a common configuration schema that all states can process. Each instance of a configuration, does however only work with a specific state, with the appropriate alphabet of non-terminal and terminal properties arranged into sections of Object Notation. The whole alphabet does not have to be used, just enough to instruct a state how to process.

A decent example of JSON that has good sections that are all parsed and applied for their purpose is the Vega Spec format. How it is parsed is described in the Parsing (parse/spec.js) area of the following link : https://github.com/trifacta/vega/wiki/Internals

My own configurations have a completely different purpose, however it is interesting to see that sections of a JSON file can be parsed and applied (in Vega's case all at once plus on updates). Most of my configurations are applied through interactivity, which is being driven by user events such as clicking or dragging. This also means that most configuration sections and whatever properties they define will need to be processed many times during a drag as trigger points are determined.

I described all this because it might be relevant to StateJS and also because just at this moment I am immersed in the activity of sorting out the roles between a FSM, the states in it, each states core execution function and its helper functions and the configuration each state can be directed/instructed by. The configuration itself has sections that terminate in simple properties, references to data elsewhere in the system or functions of different types.

All these possible sections are triggered and processed once per click or multiple times per drag using further input from the drag event itself. A second concurrent FSM would also be used to inject dynamic user choices, this injection also has to be kept separate from a given state and its configuration, to avoid mutual access to the same configuration structure from two FSM's. The first FSM states would then just look at what the second FSM produced at each trigger point. So states in the first FSM would have live momentary instructions produced by a second FSM in addition to their configuration instructions.

This is a high level view of the system I am achieving. Hope it contributes something now as I continue to work out the factors required by my FSM's. Its all about having no holds barred and form following function :)

[nickfargo]

Updated issue to formalize a grammar for the symbol definitions and how they would be resolved within selectors.

@zoomclub I think I gather the gist of what you’re describing, though the narrative form still leaves it rather abstract for me; a few specific examples would probably be helpful.

[zoomclub]

Will have a much more specific FSM structure to work with soon. Still making sure the app features are accounted for and in the process getting closer to something that StateJS seems well suited for. Exchangeable configurations can likely be handled by a prototype state. It will manage binding to the right configuration for active states that extend the prototype state. The grammar you have outlined above looks very robust, I hope to have something that can make good use of the expressiveness you have given StateJS.

On a side note, I'm watching for news on the new Famo.us render, physics and gesture engines. The grand unveiling was tonight in S.F. so there should be a lot of buzz over the next days to follow up on, along with code samples on CodePen. Looks very interesting!