A library system for abstract patterns and abstract rule-sets

[rjelliffe]

This is related to the #9 issue about abstract rules.

The current system of extends and import does not really allow libraries of abstract rules to be created.This is because an abstract rule may require a pattern variable, or even a schema variable. (I did not consider it a real problem, for the initial use case of providing simple macros to simplify repeated code.)

Also, a rule in a library may have

Suggestion: only extend abstract rules that are either in same pattern or are located in a rule library.

A library is an external document (or included under /sch:schema) containing abstract patterns and new abstract rule-sets which are not tested. They must be brought into the schema e.g. using sch:extends.

A library! ... The differences are: 1) abstract patterns and abstract rule-sets can have their own diagnostics and properties 2) abstract rule-sets can have variables (sch:let) which are injected into their parent rule (error if name clash) when used 3) libraries are part of the schema construction phase, not visible to the schema execution phase. In the spirit of minimalism, we resolve a reference to an abstract rule by looking for the simple id only, first in the current pattern then in any sch:library/sch:rule-set/sch:rule. This is a library system not a module system where the modules are part of an addressable path: no "namespaces"! ( If one rule is taken from an abstract rule-set, all its variable declarations are in-scope (i.e. injected) into the host pattern also. Similarly, its properties and diagnostics are in scope)

[AndrewSales]

@rjelliffe is it worth extending this approach to concrete patterns and rule-sets too? For ease of maintenance and composing schemas from shared libraries, for example.

[rjelliffe]

I am not sure what the use case would be: so... maybe?

To me, the general use case would be a library of XSD data types, or a library of address validators: probabky more daratypey things than structural. It would be neat if we could do the equiv of

import svg library import dcterms library

It would be wonderful if these could be integrate into package management or caching systems, so that the implementation looks in a local location otherwise downloads. What this requires is perhaps that libraries have a minor and major version number (not in namespaces) and a policy (if you request library maj.min, then you can use a cached library if same maj and

= min version number.

RICK

Cheers Rick

On Tue, 6 Feb. 2024, 20:32 Andrew Sales, @.***> wrote:

@rjelliffe https://github.com/rjelliffe is it worth extending this approach to concrete patterns and rule-sets too? For ease of maintenance and composing schemas from shared libraries, for example.

— Reply to this email directly, view it on GitHub https://github.com/Schematron/schematron-enhancement-proposals/issues/39#issuecomment-1929114077, or unsubscribe https://github.com/notifications/unsubscribe-auth/AF65KKK6L6O5KNFBH46JEXLYSH2DVAVCNFSM53A2CIS2U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOJSHEYTCNBQG43Q . You are receiving this because you were mentioned.Message ID: @.*** com>

[dmj]

I was looking into the potential of a sch:schema/sch:import element that imports global definitions of another schematron. I.e. patterns, phases, diagnostics, properties, name space declarations, variables etc.

A new top-level element for imports rather/complementary to a sch:library allows us to specify all aspects of the importing behavior, i.e. what to do with conflics.

[rjelliffe]

Would it be better to allow the existing sch:import and sch:extends to allow IDs or fragment identifiers so that you can, e.g. import the named elements.

Rick

On Fri, Feb 9, 2024 at 6:54 PM David Maus @.***> wrote:

I was looking into the potential of a sch:schema/sch:import element that imports global definitions of another schematron. I.e. patterns, phases, diagnostics, properties, name space declarations, variables etc.

A new top-level element for imports rather/complementary to a sch:library allows us to specify all aspects of the importing behavior, i.e. what to do with conflics.

— Reply to this email directly, view it on GitHub https://github.com/Schematron/schematron-enhancement-proposals/issues/39#issuecomment-1935475550, or unsubscribe https://github.com/notifications/unsubscribe-auth/AF65KKKZYEGYPVQG246SUDTYSXI3VAVCNFSM53A2CIS2U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOJTGU2DONJVGUYA . You are receiving this because you were mentioned.Message ID: @.*** com>

[AndrewSales]

Would it be better to allow the existing sch:import and sch:extends to allow IDs or fragment identifiers so that you can, e.g. import the named elements.

include and extends already do, don't they?

I was looking into the potential of a sch:schema/sch:import element that imports global definitions [...]

@dmj can you raise a separate issue for this? Thanks.

[rjelliffe]

Languages usually make a distinction between include and import: where include operations are really just text/tag substitutions and macro-parameter substitution, while import can potentially trawl through the semantics of the markup to use declarations of interest. XSD uses this approach IIRC. Basic Includes are easy to implement; imports are harder, especially where they require traversing multiple schema modules or multiple imports with potentially different overrides.

But, I think, Schematron uses a different distinction: between include and extends where the essential difference is that the include just pulls things in, while extend pulls things in expecting to vary them in some well-defined way: e.g. so that if complete schema A imports complete schema B, then every document that is invalid against B will also be invalid against A. (I think this is true??) With include, I don't think this kind of relationship needs to be a consideration.

In other words extends (=appending extra constraints) is more like XSD's restriction (i.e. reducing the number of valid documents ).

So if you are adding an import mechanism, I think it would be prudent to

1) Err on the side of simplicity of implementation, a good proxy for which is often "can it be implemented as a pre-processor easily? 2) Consider whether it allows clear(??) understanding in terms of set operations a la sch:extends.

I guess another approach might be

< sch:include href="my-library.sch" filter="/*/(sch:ns|sch:properties|sch:diagnostics)" />

Rick

On Sat, Feb 10, 2024 at 10:56 PM Andrew Sales @.***> wrote:

Would it be better to allow the existing sch:import and sch:extends to allow IDs or fragment identifiers so that you can, e.g. import the named elements.

include and extends already do, don't they?

I was looking into the potential of a sch:schema/sch:import element that imports global definitions [...] @dmj https://github.com/dmj can you raise a separate issue for this? Thanks.

— Reply to this email directly, view it on GitHub https://github.com/Schematron/schematron-enhancement-proposals/issues/39#issuecomment-1936987667, or unsubscribe https://github.com/notifications/unsubscribe-auth/AF65KKPN4NYSXEIJ4U5HISLYS5OAFAVCNFSM53A2CIS2U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOJTGY4TQNZWGY3Q . You are receiving this because you were mentioned.Message ID: @.*** com>

[dmj]

Would it be better to allow the existing sch:import and sch:extends to allow IDs or fragment identifiers so that you can, e.g. import the named elements. Rick

This would still require a specification of importing items from another ISO schematron. A pattern, rule, assertion etc. may use top-level constructs such as diagnostics, properties, variables, or functions.

I think we could discuss this separately: a. Specify an importing mechanism, i.e. how to make top-level constructs from a different schema available and b. discuss if sch:extends would be a fit or if we need a distinct schematron element.

@AndrewSales Will raise an issue on importing.

[AndrewSales]

Might be worth adding let and param to library, as well as making patterns optional, to remove the need to add a superfluous empty pattern just to maintain validity.