Variables are not always resolved to the correct type in pure mode

[bertrand-lorentz]

Following the change I proposed in PR #164, I've noticed that in certain cases, when the variable value is evaluated in the let statement, the result is not in the expected type.

For example:

    <let name="stringVar" value="'foo'"/>
    <rule context="/*">
      <assert role="ERROR" test="$stringVar eq 'bar'">error message</assert>
    </rule>

Fails with the error: Failed to evaluate XPath expression to a boolean: '$stringVar eq 'bar'' (net.sf.saxon.trans.XPathException: Values are not comparable (xs:boolean, xs:string))"

And:

    <let name="booleanVar" value="1 eq 1"/>
    <rule context="/*">
      <assert role="ERROR" test="$booleanVar eq true()">error message</assert>
    </rule>

Fails with the error: Failed to evaluate XPath expression to a boolean: '$booleanVar eq true()' (net.sf.saxon.trans.XPathException: Values are not comparable (xs:double, xs:boolean))"

This is because in PSXPathBoundSchema._evaluateVariables(), we try to evaluate the variable with each possible return type (NodeList, Boolean, String, etc.) one after the other, and stop at the first that doesn't throw an Exception.

I don't think there's a specific order of return types that would work in all cases. For example, a number can be evaluated as a boolean, and a boolean can be evaluated as a number.

It would be nice to have a way the evaluate an XPath without specifying (or implying) the return type, but that does not exist for javax.xml.xpath.XPathExpression.evaluate(). I've tried also with XPathExpression.evaluateExpression() but it does not seem to work.

The custom API in Saxon (s9api) does seem to offer this option: https://www.saxonica.com/documentation12/index.html#!xpath-api/s9api-xpath But I guess this comes with its own challenges...

Any ideas or suggestions on how to tackle this are welcome !

[phax]

Well, I think this is kind of unavoidable, as you can always have type errors in the test expressions. Eventually the change from #171 also makes the problem more visible, and you get these error messages directly inside your SVRL, as in

<svrl:schematron-output xmlns:svrl="http://purl.oclc.org/dsdl/svrl">
  <svrl:active-pattern />
  <svrl:fired-rule context="/xml" />
  <svrl:fired-rule context="/xml/owner" />
  <svrl:failed-assert location="/xml/owner[1]" test="$booleanVar eq true()" role="ERROR">
    <svrl:text>Failed to evaluate XPath expression to a boolean.
Test: '$booleanVar eq true()'
Error: Values are not comparable (xs:double, xs:boolean)</svrl:text>
  </svrl:failed-assert>
  <svrl:fired-rule context="/xml/owner" />
  <svrl:failed-assert location="/xml/owner[2]" test="$booleanVar eq true()" role="ERROR">
    <svrl:text>Failed to evaluate XPath expression to a boolean.
Test: '$booleanVar eq true()'
Error: Values are not comparable (xs:double, xs:boolean)</svrl:text>
  </svrl:failed-assert>
</svrl:schematron-output>

does that make sense to you?

[bertrand-lorentz]

I'm not sure I'm following, or maybe I'm missing something.

The way I see it, there is no type error in the schematron: the value defined for $booleanVar in the let statement is a boolean, but then using this variable in a test expression to compare it to a boolean fails. So the issue here is that it says that the values are not comparable, whereas they are both really booleans.

Thanks for adding the unit test for this issue. The correct outcome for this test should be a single failed assert, because 'foo' is not equal to 'bar', and not because the values are not comparable.

[phax]

@bertrand-lorentz sorry, my bad. I misinterpreted the question. You are of course right - the issues are unaffected. I see the problem. We have a fixed way of evaluating queries to a certain type:

1. Node list
2. Number
3. Boolean
4. String
5. Node

Whatever is matched first, wins. In the XSLT version, the let is mapped to an xsl:param without an as attribute:

  <xsl:param name="abc" select="1 eq 1" />

In XSLT it uses the supplied type, and that is exactly what we're lacking...

So the only reasonable solution I could come up with, was some Saxon specific magic. I checked and all unit tests work without problems. May be you can also test that with your code base, so that we can eventually extend the Saxon specific type handling over time. Please let me know what you think.

[bertrand-lorentz]

Thanks for the improvement, it seems to be working pretty well. For example, Schematron test expressions that call the concat($stringVar1, $stringVar2) function with 2 variables that are strings now work.

I did notice a problem with a variable declared as follows: <let name="docVar" value="fn:doc('http://localhost/foo.xml')"/>

I got the following error:

531071 [main] WARN com.helger.schematron.pure.xpath.XPathEvaluationHelper$SaxonEvaluator - Unknown Saxon type: net.sf.saxon.pattern.NodeKindTest
682417 [main] ERROR com.helger.schematron.pure.errorhandler.LoggingPSErrorHandler - [error] in [PSRule] @ /path/to/schematron.sch Failed to evaluate XPath expression 'net.sf.saxon.xpath.XPathExpressionImpl@2484f433' for variable 'docVar'

The fn:doc() function returns a "document node". I tried to see how to handle it, but I don't know enough about Saxon to figure out what to do with a NodeKindTest.

In any case, the fn:doc() is in some rather experimental Schematron code, so not a big deal.

Solving the problem for variables with simple types (string, number, boolean, etc.) is already a great improvement.

[phax]

@bertrand-lorentz thanks for testing and providing feedback. I am well aware, that not all combinations and types are known and supported but I am positive, that this list will be completed over time. I now added support for the NodeKindTest as well. Nevertheless I think it's a good point to release the next minor update and lets see, how the community reacts to it.

Thanks as usual for your valuable contributions and pointing me into the right direction :)