Documenting Metapath Syntax

[aj-stein-nist]

User Story:

As a Metaschema developer, to ensure I can implement a Metaschema-enabled library and/or tools that must completely implement Metapath for use in constraint targets and index targets, I want a specification document regarding the syntax of Metapath, supported axes for queries, and required functions.

Goals:

• [ ] Document Metapath syntax
  • [ ] Core syntax
  • [ ] Axes for queries
  • [ ] Required functions
  • [ ] Highlight its relationship to XPath syntax and its differences and what subset of functionality the things above support

Dependencies:

None.

Acceptance Criteria

• [ ] All website and readme documentation affected by the changes in this issue have been updated. Changes to the website can be made in the docs/content directory of your branch.
• [ ] A Pull Request (PR) is submitted that fully addresses the goals of this User Story. This issue is referenced in the PR.
• [ ] The CI-CD build process runs without any reported errors on the PR. This can be confirmed by reviewing that all checks have passed in the PR.

{The items above are general acceptance criteria for all User Stories. Please describe anything else that must be completed for this issue to be considered resolved.}

[david-waltermire]

Looking at the [XPath syntax](), Metapath is expected to support:

2 Basics

• 2.1 Expression Context
  • 2.1.1 Static Context
  • 2.1.2 Dynamic Context
• 2.2 Processing Model
  • 2.2.1 Data Model Generation
  • 2.2.2 Schema Import Processing
  • 2.2.3 Expression Processing
    • 2.2.3.1 Static Analysis Phase
    • 2.2.3.2 Dynamic Evaluation Phase
  • 2.2.4 Consistency Constraints
• 2.3 Error Handling
  • 2.3.1 Kinds of Errors
  • 2.3.2 Identifying and Reporting Errors
  • 2.3.3 Handling Dynamic Errors
  • 2.3.4 Errors and Optimization
• 2.4 Concepts
  • 2.4.1 Document Order
  • 2.4.2 Atomization
  • 2.4.3 Effective Boolean Value
  • 2.4.4 Input Sources
  • 2.4.5 URI Literals
  • 2.4.6 Resolving a Relative URI Reference
• 2.5 Types
  • 2.5.1 Predefined Schema Types
  • 2.5.2 Namespace-sensitive Types
  • 2.5.3 Typed Value and String Value
  • 2.5.4 SequenceType Syntax
  • 2.5.5 SequenceType Matching
  • 2.5.5.1 Matching a SequenceType and a Value
  • 2.5.5.2 Matching an ItemType and an Item
  • 2.5.5.3 Element Test
  • 2.5.5.4 Schema Element Test
  • 2.5.5.5 Attribute Test
  • 2.5.5.6 Schema Attribute Test
  • 2.5.5.7 Function Test
  • 2.5.5.8 Map Test
  • 2.5.5.9 Array Test
  • 2.5.6 SequenceType Subtype Relationships
  • 2.5.6.1 The judgement subtype(A, B)
  • 2.5.6.2 The judgement subtype-itemtype(Ai, Bi)
  • 2.5.7 xs:error
• 2.6 Comments

3 Expressions

• 3.1 Primary Expressions
  • 3.1.1 Literals
  • 3.1.2 Variable References
  • 3.1.3 Parenthesized Expressions
  • 3.1.4 Context Item Expression
  • 3.1.5 Static Function Calls
  • 3.1.5.1 Evaluating Static and Dynamic Function Calls
  • 3.1.5.2 Function Conversion Rules
  • 3.1.5.3 Function Coercion
  • 3.1.6 Named Function References
  • 3.1.7 Inline Function Expressions
  • 3.1.8 Enclosed Expressions
• 3.2 Postfix Expressions
  • 3.2.1 Filter Expressions
  • 3.2.2 Dynamic Function Calls
• 3.3 Path Expressions
  • 3.3.1 Relative Path Expressions
  • 3.3.1.1 Path operator (/)
  • 3.3.2 Steps
  • 3.3.2.1 Axes
  • 3.3.2.2 Node Tests
  • 3.3.3 Predicates within Steps
  • 3.3.4 Unabbreviated Syntax
  • 3.3.5 Abbreviated Syntax
• 3.4 Sequence Expressions
  • 3.4.1 Constructing Sequences
  • 3.4.2 Combining Node Sequences
• 3.5 Arithmetic Expressions
• 3.6 String Concatenation Expressions
• 3.7 Comparison Expressions
  • 3.7.1 Value Comparisons
  • 3.7.2 General Comparisons
  • 3.7.3 Node Comparisons
• 3.8 Logical Expressions
• 3.9 For Expressions
• 3.10 Let Expressions
• 3.11 Maps and Arrays
  • 3.11.1 Maps
  • 3.11.1.1 Map Constructors
  • 3.11.1.2 Map Lookup using Function Call Syntax
  • 3.11.2 Arrays
    • 3.11.2.1 Array Constructors
    • 3.11.2.2 Array Lookup using Function Call Syntax
  • 3.11.3 The Lookup Operator ("?") for Maps and Arrays
    • 3.11.3.1 Unary Lookup
    • 3.11.3.2 Postfix Lookup
• 3.12 Conditional Expressions
• 3.13 Quantified Expressions
• 3.14 Expressions on SequenceTypes
  • 3.14.1 Instance Of
  • 3.14.2 Cast
  • 3.14.3 Castable
  • 3.14.4 Constructor Functions
  • 3.14.5 Treat
• 3.15 Simple map operator (!)
• 3.16 Arrow operator (=>)

[david-waltermire]

We need to figure out where to organize Metapath specification details in the Metaschema specification website. Any ideas?

[wendellpiez]

Having the outline is super useful.

Separately I am going to have to document a current limitation on the InspectorXSLT, namely that paths only work that conform additionally to these requirements: https://qt4cg.org/specifications/xslt-40/Overview.html#node-patterns

And this does relate to a semantic question, namely what does it mean for a constraint @target expression returns not nodes to which a constraint can apply, but something else?

For example, do these work or are they errors (if so what), no-ops or something else?

<has-cardinality target="/*/name()" max-occurs="2"/>

<expect target="42" test=". gt 40"/>

<index name="numbers" target="1 to 100">
  <key-field select="@value"/>
</index>

Of these, I'd call the first an innocuous mistake (why count the names instead of the things), the second meaningless (42 is always greater than 40) and the third an error (because numeric values don't have 'value' flags to index them by).

These problems and more could be avoided by restricting Metapath in @target to paths returning sequences of nodes.

Bringing it all the way to current functionality in the InspectorXSLT - which handles @target by producing templates to match the implicated nodes - would require an even stricter limit, namely to the XSLT pattern syntax linked above.

This would offer other significant advantages as well. For example, the XPath axes following-sibling:: and preceding-sibling:: are somewhat curious in a language for addressing JSON - and pattern syntax includes only XPath axes that look down to branches the tree, not up or over.

expect/@test does not pose at all the same problems in implementation as @target does - mainly since more or less anything can be reduced into a Boolean value for @test, but @target has to have nodes to serve as context for evaluation.

[aj-stein-nist]

We need to figure out where to organize Metapath specification details in the Metaschema specification website. Any ideas?

I am fine with putting Metapath under the syntax section, as it came up during discussion. It could be both considered as something separate of Metaschema syntax or something separate embedded within certain Metaschema elements, but I am not picky and will wait from more feedback from this group and the larger community.

Are we going to spell out every section of the XPath 3.0 specification and copy it more or less, have a page on the site specifically call out this outline and define the scope with links to relevant page (so cross out items are a not-supported list and not crossouts), and third more complex section is custom written sections where the content is different?

[aj-stein-nist]

These problems and more could be avoided by restricting Metapath in @target to paths returning sequences of nodes.

How do we know what a path is when we have a evaluation focus context? Is target="." referring to a path unconditionally or does it depend on the content? If it is a constraint for a flag or field that could be but how do we know that in a way a Metaschema processor could know?

[wendellpiez]

These are excellent questions and I can walk you through how it works in my implementation, with the (XML/XPath-based) assumptions that implies. (I.e. 'how it could/would work in XPath' not 'how it should work (definitively) in Metaschema'. This is because it hinges crucially on the notion of relative paths evaluated in context.

The question is a little different for the different constraints, as well - what happens with matches or expect is a bit different from has-cardinality, etc. (Whether @target names something that is counted, or serves as a context for further evaluation.)

But to jump past a longer explanation, keep in mind that where a constraint (rule) is defined in a metaschema does provide context in the form of this rule's location within a definition (local or global). Outside such placement (i.e., written as a 'standoff rule') the @target will implicitly be reaching always from the root. (While nothing I say means anything without demo samples, shared across implementations.)