Attribute node selection with "kind test" attribute() or attribute(foo) doesn't work

[martin-honnen]

The following two test cases fail:

       [Fact]
        public void FullPathWithAttributeSelection()
        {
            var nav = GetTodoListDoc().CreateNavigator();
            var result1 = nav.XPath2Evaluate("count(/todo-list/todo-item/@id)");
            var result2 = nav.XPath2Evaluate("count(/todo-list/todo-item/attribute(id))");

            Assert.Equal(result1, result2);
        }

        [Fact]
        public void AttributeSelection()
        {
            var nav = GetTodoListDoc().CreateNavigator();
            var result1 = nav.XPath2Evaluate("count(//@id)");
            var result2 = nav.XPath2Evaluate("count(//attribute(id))");

            Assert.Equal(result1, result2);
        }

The attempts using attribute() or attribute(id) seem to select 0 nodes, somehow it looks as if the XPath 2 implementation doesn't seem to recognize the syntax correctly.

[martin-honnen]

As far as I understand it currently, the problem arises due to https://github.com/StefH/XPath2.Net/blob/master/src/XPath2/XPath.y#L619 implementing the rules

AbbrevForwardStep
   : '@' NodeTest
   {
       $$ = new PathStep($2, XPath2ExprType.Attribute);
   }
   | NodeTest
   {
       $$ = new PathStep($1, XPath2ExprType.Child);
   }
   ;

so only with @ a PathStep of type XPath2ExprType.Attribute is created, anything else, like text() or comment() or attribute()orattribute(foo)ends up as aPathStepofXPath2ExprType.Child, end that is obviously completely wrong for theattribute()orattribute(foo)` node tests.

[martin-honnen]

@StefH , is that syntax of the grammar productions in XPath.y and its interaction with generated C# code documented somewhere? I am kind of struggling to understand how to change/rewrite e.g. that rule matching any NodeTest to an XPathStep of type XPath2ExprType.Child to perform any PathStep creation later on when the rule of a particular KindTest is matched.

[martin-honnen]

It seems the "jay" software is at https://www.cs.rit.edu/~ats/projects/lp/doc/jay/package-summary.html although it doesn't have a lot of explications, rather it seems the user is familiar with or willing to dive into literature on "yacc" which is using grammars mixed with C actions while "yay" allows using grammars mixed with C# actions (or Java). One tutorial on yacc actions is https://www.ibm.com/docs/en/aix/7.1?topic=information-yacc-actions.

Still not sure which parts of the C# classes in the XPath2 package for parsing and AST are hand-written and which tool generated.

I managed to get some way to get attribute() kind selection working by moving the code to create a PathStep to the rules

AttributeTest
   : ATTRIBUTE '(' ')'
   {
      $$ = new PathStep(XmlQualifiedNameTest.New(null, null), XPath2ExprType.Attribute);
   }
   | ATTRIBUTE '(' AttributeOrWildcard ')'
   {
      $$ = new PathStep(new SequenceType(XmlTypeCode.Attribute, (XmlQualifiedNameTest)$3), XPath2ExprType.Attribute);
   }
   | ATTRIBUTE '(' AttributeOrWildcard ',' TypeName ')'
   {
      XmlSchemaObject xmlType;
      CoreFuncs.TryProcessTypeName(context, (string)$5, true, out xmlType);
      $$ = new PathStep(new SequenceType(XmlTypeCode.Attribute, (XmlQualifiedNameTest)$3, (XmlSchemaType)xmlType), XPath2ExprType.Attribute);      
   }
   ;

but I had to stop creating a PathStep in the right hand side of the rule for `AbbrevForwardStep : NodeTest (which did

AbbrevForwardStep
   : '@' NodeTest
   {
       $$ = new PathStep($2, XPath2ExprType.Attribute);
   }
   | NodeTest
   {
       $$ = new PathStep($1, XPath2ExprType.Child);
   }
   ;

) before and unfortunately so far my attempts break other unrelated expressions, kind of down from 86 to 84 % of the test cases in the test suite passing.

[StefH]

If you go to the \dependencies\jay folder and run run-jay.cmd, the changes done in the .py file are converted into c#.

And I discovered that you can debug some code, it seems that the .py and .cs code are linked.

However, only the file xpath.cs is generated, the rest of the code is normal code.

[StefH]

I did contact the original author from this library, and he stated that you can only use attribute(id) if you prefix it with attribute::.

See linked PR for details.

[martin-honnen]

Is that argument based on the XPath 2.0 specification? I don't see any such restriction in the rules. A ForwardStep can be an AbbrevForwardStep which can be a NodeTest which can be an AttributeTest.

https://www.w3.org/TR/xpath20/#prod-xpath-ForwardStep states

[29]    ForwardStep    ::=      (ForwardAxis NodeTest) | AbbrevForwardStep

https://www.w3.org/TR/xpath20/#abbrev states

[31]    AbbrevForwardStep      ::=      "@"? NodeTest

and

https://www.w3.org/TR/xpath20/#prod-xpath-NodeTest states

[35]    NodeTest       ::=      KindTest | NameTest

and https://www.w3.org/TR/xpath20/#prod-xpath-KindTest states

[54]    KindTest       ::=      DocumentTest
| ElementTest
| AttributeTest
| SchemaElementTest
| SchemaAttributeTest
| PITest
| CommentTest
| TextTest
| AnyKindTest
[55]    AnyKindTest    ::=      "node" "(" ")"
[56]    DocumentTest       ::=      "document-node" "(" (ElementTest | SchemaElementTest)? ")"
[57]    TextTest       ::=      "text" "(" ")"
[58]    CommentTest    ::=      "comment" "(" ")"
[59]    PITest     ::=      "processing-instruction" "(" (NCName | StringLiteral)? ")"
[60]    AttributeTest      ::=      "attribute" "(" (AttribNameOrWildcard ("," TypeName)?)? ")"
[61]    AttribNameOrWildcard       ::=      AttributeName | "*"
[62]    SchemaAttributeTest    ::=      "schema-attribute" "(" AttributeDeclaration ")"
[63]    AttributeDeclaration       ::=      AttributeName
[64]    ElementTest    ::=      "element" "(" (ElementNameOrWildcard ("," TypeName "?"?)?)? ")"
[65]    ElementNameOrWildcard      ::=      ElementName | "*"
[66]    SchemaElementTest      ::=      "schema-element" "(" ElementDeclaration ")"