p5odds.odd is too restrictive

[martindholmes]

Ron is adding new features to the jTEI ODD file to provide support through the schema for Oxygen's new QuickFix feature. These are elements in the sqf namespace which provide hints for Oxygen users to help them resolve the problem flagged by the constraint:

<constraint>
                 <sch:rule context="tei:titleStmt">
                 <sch:assert test="tei:title[@type = 'main']" sqf:fix="type.add">
                     A title of type "main" is required in <sch:name/>.
                   </sch:assert>
                 <sqf:fix id="type.add">
                   <sqf:description>
                    <sqf:title>Add a @type='main' attribute to the first title.</sqf:title>
                  </sqf:description>
                   <sqf:add match="tei:title[not(@type='main')][1]" node-type="attribute" target="type">main</sqf:add>
                  </sqf:fix>                  
                 </sch:rule>
               </constraint>

This would be valid according to the standard P5 schema, but it isn't valid against p5odds.odd, the more restrictive version we use for validating ODD files in the P5 tree. We would like to see this changed. The two options would be:

• Remove the extra strictness on the content model of <constraint> so that anything allowed by the standard tei_all is allowed in p5odds; or
• make a special case of this sqf content, and allow it explicitly in p5odds.odd.

[lb42]

I assume you don';t actually mean "anything allowed by the standard tei_all is allowed in p5odds;" if that IS what you want, why not just use tei_all ? So what DO you mean? The ability to add things from an additional namespace?

[martindholmes]

The problem is the TEI build process which validates the jTEI ODD using a schema generated from p5odds. We could certainly tweak the build process so that it ignores this particular ODD or validates it with tei_all, but I assume there is some value in our having a tighter schema for validating the P5 products than tei_all provides. So, to clarify:

I don't think it's unreasonable to include the sqf enhancements in the jTEI schema, so I think the p5odds.odd content model for <constraint> should allow that. I don't really mind how that's done, but I think it's best done by changing the content model in p5odds.odd, not by arbitrarily removing the jTEI schema from the validation process that applies to the rest of our P5 products.

[lb42]

This is the anyXML problem in a different guise. p5odds.odd redefines <constraintSpec so as to permit any text or elements from either of the namespaces http://www.ascc.net/xml/schematron and http://purl.oclc.org/dsdl/schematron . Adding a third possibility, of any element from whatever namespace it is that corresponds with this "sqf" prefix, seems a perfectly reasonable (and easy) fix to me. Expressing the required content model in pure odd will of course be a different challenge.

[lb42]

So easy in fact that I've now added it. Please test !

[martindholmes]

Thanks Lou! I'll ask Ron to resubmit his changes.

[martindholmes]

That didn't work, as you can see. But the sqf stuff appears inside a <sch:rule>, not as a sibling of it, so I'm not sure why it's invalid in the first place, unless the effect of the existing macros is to force the entire subtree to be in the specified namespace. Still trying to figure this out.

[lb42]

I hadn't noticed that the sqf stuff is nested inside a sch element. In which case your problem is with the schematron schéma and nothing to do with tei methinks.

[martindholmes]

This is very strange. It's clearly not trying to validate the Schematron itself, since if that were the case it would error out on encountering the @sqf:fix="type.add" attribute on the <sch:assert> element. But it won't allow sqf: elements lower in the tree. Any ideas, anyone?

[martindholmes]

If it were being validated with the Schematron schema, then why would the regular P5 schema (which uses macro.anyXML) work?

But if that is the case, we'll have to either:

1. Change the content model in p5odds.odd to use macro.anyXML, or
2. Special-case the jtei ODD and validate it with the p5 schema rather than the p5odds schema.

Which do you prefer?

[lb42]

Because Macro.anyXML accepts as valid anything except two spec ified name spaces whereas its redefinition here accepts only things from specific name spaces and therefore has to validate against the schemas associated with them. Check what it says in the o n vd l. I think your quickest solution is to just remove the redefinition of macro.anyXML from the odd.

[martindholmes]

You mean the redefinition of the content model of <constraint>, surely?

[rvdb]

If I'm understanding the issue correctly, I think the current validation problem is consistent with the definition of anyISOSchematron:

    <macroSpec ident="anyISOSchematron" mode="add">
      <content>
        <element xmlns="http://relaxng.org/ns/structure/1.0">
          <nsName ns="http://purl.oclc.org/dsdl/schematron"/>
          <zeroOrMore>
            <choice>
              <attribute>
                <anyName/>
              </attribute>
              <ref name="anyISOSchematron"/>
              <text/>
            </choice>
          </zeroOrMore>
        </element>
      </content>
    </macroSpec>

This clearly limits the contents to elements form the http://purl.oclc.org/dsdl/schematron namespace. Since the Schematron Quick Fix (SQF) elements are defined as "an extension of the ISO standard Schematron" (see http://www.schematron-quickfix.com/index.html), I think the following steps could solve the problem:

1. remove anySchematronQuickFix from the definition of <constraint> again
2. add anySchematronQuickFix or the SQF namespace to the definition of anyISOSchematron

Intuitively, I would think following definition of the anyISOSchematron macro could be sufficient:

    <macroSpec ident="anyISOSchematron" mode="add">
      <content>
        <element xmlns="http://relaxng.org/ns/structure/1.0">
          <choice>
            <nsName ns="http://purl.oclc.org/dsdl/schematron"/>
            <nsName ns="http://www.schematron-quickfix.com/validator/process"/>
          </choice>
          <zeroOrMore>
            <choice>
              <attribute>
                <anyName/>
              </attribute>
              <ref name="anyISOSchematron"/>
              <text/>
            </choice>
          </zeroOrMore>
        </element>
      </content>
    </macroSpec>

Then, the anySchematronQuickFix macro could just be removed. Yet, the <choice> element containing the <nsName> elements seems to be ignored when p5odds.odd is transformed to a Relax NG XML schema. Don't know if this is a bug in the XSLT stylesheets, or if there's anything wrong with this <choice> element. It is valid when pasted in the generated Relax NG schema, though.

Another possibility is to preserve the anySchematronQuickFix macro and refer to it in the anyISOSchematron definition:

    <macroSpec ident="anyISOSchematron" mode="add">
      <content>
        <element xmlns="http://relaxng.org/ns/structure/1.0">
          <nsName ns="http://purl.oclc.org/dsdl/schematron"/>
          <zeroOrMore>
            <choice>
              <attribute>
                <anyName/>
              </attribute>
              <ref name="anyISOSchematron"/>
              <ref name="anySchematronQuickFix"/>
              <text/>
            </choice>
          </zeroOrMore>
        </element>
      </content>
    </macroSpec>

The latter option does generate a valid p5odds.rng schema, which in turn validates the up-to-now problematic tei_jtei.odd file containing the SQF elements. I've attached this version as p5odds-tweak.odd.

A disclaimer, finally: this all is tested locally, but I think these are the issues at play in the build process.

[martindholmes]

Hi Ron. Thanks for debugging this. Could you commit your working change to p5odds.odd? I think I like it better than the generic one that doesn't work anyway.

[martindholmes]

Thanks Ron, it looks like that worked:

http://teijenkins.hcmc.uvic.ca/job/TEIP5/2313/

Are we all happy with that change? Shall I close the ticket?

[rvdb]

Hi Martin, I've just tested the generated jTEI schema, and the Schematron Quick Fixes are functional.

Unless there's a more elegant solution to express this in p5odds.odd (I'm still not sure what would be wrong with the definition of both the http://purl.oclc.org/dsdl/schematron and http://www.schematron-quickfix.com/validator/process namespaces in the anyISOSchematron macro), I'm happy.

[martindholmes]

Hi Ron. I think the fact that the change to the anyISOSchematron macro doesn't work is enough for us to leave it alone for now; it might be due to a bug in the XSLT, but Pure ODD is coming, and presumably bringing with it new ways to express these macros, so we might as well wait until we have that, and then make sure the associated XSLT works there. If no-one has any further comment in a couple of days, I'll close this ticket.

[lb42]

Thanks ron. Your solution looks fine to me. Good to know I was barking up the right tree even if my solution was incomplete!

[hcayless]

Can this be closed now?

[lb42]

Yes.