search

xspec / xspec

XSpec is a unit test and behaviour-driven development (BDD) framework for XSLT, XQuery, and Schematron.
https://github.com/xspec/xspec/wiki
MIT License
112 stars 31 forks source link

xsl:sequence Code Coverage Non-Conformance #1945

Closed birdya22 closed 1 month ago

birdya22 commented 2 months ago

Current Non-Conformance

The xslt-code-coverage-behavior-by-element.md file says that xsl:sequence elements should be marked based on their child elements. The coverage-report.xsl in 3.0.3 marks them as 'missed' because there is no hit element in the XSpec trace file and there is no rule in coverage-report.xsl to process them.

Changes in coverage-report.xsl

A rule needs to be added to coverage-report.xsl as follows for xsl:sequence: If node has no children, mark it as 'unknown'. If the trace data has a "hit" element for a child of this node, then mark this node as a 'hit'. Otherwise, mark it as 'missed'.

Changes in test/end-to-end/cases-coverage/expected

In the coverage report xsl-sequence-01-coverage.html the first xsl:sequence element will be marked as 'ignored' rather than 'missed'. The second xsl:sequence element will be marked as 'hit' rather than 'missed'.

In the coverage report xsl-analyze-string-01-coverage.html the 1 xsl:sequence element will be marked as 'ignored' rather than 'missed'. In the coverage report xsl-on-non-empty-01-coverage.html the 1 xsl:sequence element will be marked as 'ignored' rather than 'missed'.

The change will also have an effect on the xsl:sequence elements in the xsl-map-01-coverage.html coverage report. Once a decision has been made on the rules for xsl:map and xsl:map-entry this issue will be updated with details of any changes to the xsl:sequence elements.

galtm commented 1 month ago

Using getLocation (#1971) in Saxon 12.4 caused an <xsl:sequence select="@nonexistent-attribute" /> element to have a correct hit in the trace data, though other xsl:sequence elements with a select attribute do not have a hit in their trace data. Maybe we should use a kind of hybrid rule like this:

  1. If the trace data has a hit for this element, mark it as 'hit'.
  2. Otherwise, follow Use Child Data (a.k.a. Use Descendant Data) rule.

That way, the report won't mark an element as 'unknown' when the trace data has a hit.

birdya22 commented 1 month ago

Yes. The trace isn't related to the xsl:sequence element but to the xsl:on-non-empty element I think, which would explain why other xsl:sequence elements aren't hit. . The trace has a class of net.sf.saxon.expr.instruct.ConditionalBlock

galtm commented 1 month ago

Changes in test/end-to-end/cases-coverage/expected

In the section above about changes in expected reports, I believe you wrote 'ignored' where you meant 'unknown'. The section of this issue description about XSLT changes says 'unknown', which also aligns with how we've been discussing 'unknown' as a status to use when trace data isn't sufficient.