Prepare XML types for expressions

[WJSchakel]

To run different scenarios from XML, certain settings/parameters need to be changed between scenarios. As anything might need to be changed between scenarios, a flexible approach is required. Under the <Scenarios> tag there will be a list of input parameters, with default value. Each scenario can set a different value on input parameters. To generically use input parameters in other places in XML, values can be defined as expressions that refer to input parameters. Expressions are given between { }. A simple case is for example: {pTruck}, which will simply evaluate to the value of input parameter pTruck. More complicated expressions are possible, but that's not relevant here.

To be able to define anything in the normal manner, or using an expression, all XML types need to be enhanced. For all types that are an extension of xsd:string and are bounded by a pattern, this pattern needs to be extended. For an existing pattern abc:

<xsd:pattern value="\{[^{}]+\}|(abc)">

This means that any string between { } is also allowed, except that { and } are not allowed within the string, or the regular pattern for the type applies.

For any type that uses base xsd types, such as xsd:double, an xsd:union restriction is suitable. The union will be of xsd:double (or some other base type) and ots:Expressions. The type ots:Expression will be a new type that extents xsd:string with:

<xsd:pattern value="\{[^{}]+\}">

This means that the string must start with { and end with }, and may not have any of these braces between them.

[WJSchakel]

Types that are restricted by an enumeration, can be defined as:

  <xsd:simpleType name="LeftRightType">
    <xsd:union memberTypes="ots:Expression">
      <xsd:simpleType>
        <xsd:restriction base="xsd:string">
          <xsd:enumeration value="L" />
          <xsd:enumeration value="LEFT" />
          <xsd:enumeration value="R" />
          <xsd:enumeration value="RIGHT" />
          <xsd:enumeration value="CLOCKWISE" />
          <xsd:enumeration value="COUNTERCLOCKWISE" />
        </xsd:restriction>
      </xsd:simpleType>
    </xsd:union>
  </xsd:simpleType>

[WJSchakel]

Changing e.g. xsd:double to ots:double (which also allows an expression) causes values to become String in the generated code. To aid parsing we can enhance the use of adapters. Adapters will need to unmarshal towards java types that either contain an expression, or a value. For this we use ExpressionType:

public class ExpressionType<T>
{
    private final T value;

    private final String expression;

    public ExpressionType(final T value);

    public ExpressionType(final String expression);

    public T get(final InputParameters inputParameters);

    public boolean isExpression();

    public String getExpression();

    public String getBracedExpression();

    public T getValue();
}

Because bindings don't support generics, extensions of this class are required for used XML types, e.g.:

public class DoubleType extends ExpressionType<Double>
{
    public DoubleType(final Double value);

    public DoubleType(final String expression);
}

An example binding for ots:double would be:

    <xjc:javaType name="org.opentrafficsim.xml.bindings.types.DoubleType" xmlType="ots:double"
      adapter="org.opentrafficsim.xml.bindings.DoubleAdapter" />

It's adapter would become:

public class DoubleAdapter extends ExpressionAdapter<Double, DoubleType>
{

    /** {@inheritDoc} */
    @Override
    public DoubleType unmarshal(final String field)
    {
        if (isExpression(field))
        {
            return new DoubleType(trimBrackets(field));
        }
        return new DoubleType(Double.valueOf(field));
    }

}

Here, ExpressionAdapter is a class that defaults to marshaling using the wrapped expression, or toString() on the wrapped value.

The XML parsing code can first create the input parameters for a specific scenario, give this along through the various parsing methods, and use ExpressionType.get(InputParameters) to obtain the right value, whether that is from an expression (using input parameters or not), or just a wrapped value.

[WJSchakel]

Some elements appear at one place and are not defined with a type. Yet it may be desirable to transform the XML element to a java type. For such elements a different binding is required (next to the jaxb:globalBindings tag). For example:

  <jaxb:bindings schemaLocation="ots-model.xsd">
    <jaxb:bindings node="//xsd:element[@name='Synchronization']">
      <jaxb:property>
        <jaxb:baseType>
          <xjc:javaType name="org.opentrafficsim.xml.bindings.types.SynchronizationType"
            adapter="org.opentrafficsim.xml.bindings.SynchronizationAdapter" />
        </jaxb:baseType>
      </jaxb:property>
    </jaxb:bindings>
  </jaxb:bindings>

This involves ots-model.xsd, for which multiple bindings can be defined in the bindings tag. Above, one such binding is defined for any xsd:element with attribute name being Synchronization. In case of name-clashes, this can be further specified following the xsd types. E.g. "//xsd:element[@name='MandatoryIncentives']/xsd:complexType/xsd:sequence/xsd:element[@name='Incentive']".

Additional bindings tags can be specified for other schema's, next to a tag is in the above example. More info can be found at: https://docs.oracle.com/cd/E17802_01/webservices/webservices/docs/1.5/tutorial/doc/JAXBUsing4.html.

[WJSchakel]

The following was done to prepare for the use of expressions:

• All types, both in XML and how they are automatically parsed, allow expressions.
• Throughout the further parsing, get(InputParameters) is used on all types. Types all extend ExpressionType<T>.
• There are a few fields that remain xsd:string, mostly ID fields (not those that are actually referring to the ID of another element). These fields are static and can't be dynamically defined with an expression.
• There are a few type adapters with specific checks, but for most types more generic adapters are used. For example, ots:double, ots:DoubleUnitInterval, ots:DoublePositive and ots:DoublePositiveInclusive are all translated to DoubleType using DoubleAdapter. More adapters with specific checks can be added later.
• Units are now parsed using an adapter.
• Due to the previous bullet, ParseDistribution now has a single generalized method to parse ContinuousDistDoubleScalar.Rel to which the right unit is provided. This replaces a slew of methods for each unit.
• Transformer class deleted. Some of its methods moved to ParseUtil. Other methods were no longer required.
• XML types describing types (LaneType etc.) may now inherit from XML types HierarchicalType or GtuCompatibleInfraType (which itself extends HierarchicalType).
• DefinitionsParser is generalized; parsing of different definition types has a lot of overlap that is easily captured in one method. Parts of the parsing are done if the parsed definition is a HierarchicalType (get from OTS defaults, get from Constructor(String, Parent) or from Constructor(String)), or GtuCompatibleInfraType (parse compatibilities). Using a consumer that is specifically provided when parsing a certain type, storing the definitions in specific maps or similar actions are also done.
• Cost (LinkWeight) added to ShortestRoute.
• In the model parser, input parameters were used to overwrite model parameter values. This code was removed. Usage of the input parameters will work only through expressions.
• Split up code in OdParser.parseDemand(...) over several methods; the method was incredibly long.

What remains is:

• Use Djutils expression evaluator in ExpressionType.get(InputParameters).
• Redefine what InputParameters exactly contains, based on scenario's.
• Add adapters with additional checks on types (or add check in additional types).
• Adapt editor to changes.

[WJSchakel]

Added DoublePositiveAdapter, DoublePositiveInclusiveAdapter, DoubleUnitIntervalAdapter, PositiveDurationAdapter, PositiveIntegerAdapter, PositiveTimeAdapter and SpaceAdapter.

[WJSchakel]

• Editor indicates values, attributes or elements with an expression with a light yellow background color.
• Invalid value messages now deal with expressions and xsd:union.
• xsd:keyref and xsd:unique are now checked ignoring expression values. This does not comply with official XML validation.

[averbraeck]

Potential solutions to the particular problem of not complying with the XML validation are discussed in issue #81