Duplicate attributes in odd → rng conversion with Release 7.56.0

nivaca commented 3 months ago

When running teitorng to convert an ODD schema to a RNG one, using the latest version (7.56.0) I get loads of "duplicate attribute" errors, e.g.

Description | duplicate attribute "lang" from namespace "http://www.w3.org/XML/1998/namespace"

I am running it standalone (from the bin/ directory in the latest cloned repo) and from the latest Oxygen XML Editor conversion utility (which I was told also uses v7.56.0) with the same issues.

I do not have this problem with v7.55.0 (run in either way).

My source ODD can be found here: https://github.com/nivaca/smtrad-schema/blob/main/smtrad.xml in case it is useful.

lb42 commented 3 months ago

I can confirm this: using your ODD I get the same problem. Also, the generated schema does not specify a root element, which seems odd (ha ha)

lb42 commented 3 months ago

Something is definitely broken. Produce a relaxng schema from just about any odd (i tried tei_bare) and it contains duplicate declarations for @calendar and no root element. Is there no regression testing for stylesheet releases?

pdaengeli commented 2 months ago

We also ran into this unexpected proliferation of RNG contents and had a closer look (on the surface, not on the level of the generating stylesheets).

Here is an MRE based on smtrad.xml:

ODD

```xml SMTrad MRE

Anglo-Saxon Anglo-Saxon lexeme Treats the quote as a lexeme ```

The three referenced modules in conjunction with the two element specifications result in the generation of duplicate attribute references and attributes in the RNG:

rng

```xml (foreignforeign) identifies a word or phrase as belonging to some language other than that of the surrounding text.identifies a word or phrase as belonging to some language other than that of the surrounding text. [3.3.2.1. Foreign Words or Expressions3.3.2.1. Foreign Words or Expressions] ❌ ❌ ❌ ❌ ❌ ❌ ❌ ❌ ❌ ❌ ❌ (language) indicates the language of the element content using a tag generated according to BCP 47. ang (Anglo-Saxon) Anglo-Saxon ❌ ❌ (language) indicates the language of the element content using a tag generated according to BCP 47. ❌ ❌ ang ❌ (Anglo-Saxon) Anglo-Saxon ❌ ❌ ```

Furthermore, the intermediary .odd.processedodd is invalid due to misplaced and spurious elements. This suggests that the cause of the faulty RNG lies in the first part of the ANT task (odd2odd.xsl).

.odd.processedodd

```xml foreign 외국어 外文 étranger extranjero straniero fremd identifies a word or phrase as belonging to some language other than that of the surrounding text. 주변 텍스트와 다른 언어로 사용된 단어나 구를 표시한다. 標明所屬語言不同於週遭文字的字詞。記述のベースで使われている言語と異なる言語による語句を定義する。 reconnaît un mot ou une expression comme appartenant à une langue différente de celle du contexte. identidica una palabra o frase perteneciente a otra lengua distinta de la del resto del texto en que se incluye. identifica una parola o un sintagma appartenente ad una lingua diversa da quella del testo circostante. identifiziert ein Wort oder eine Phrase, die zu einer anderen Sprache gehört, als der umgebende Text. Anglo-Saxon Anglo-Saxon This is heathen Greek to you still? Your lapis philosophicus? Et vobis messieurs, Ce ne seroyt que bon que nous rendissiez noz cloches...

Pendant ce temps-là, dans le bureau du rez- de-chaussée, les secrétaires faisaient du hulla-hoop.

整天窩在家裏看卡通打電動，你是御宅族嗎?

The global xml:lang attribute should be supplied for this element to identify the language of the word or phrase marked. As elsewhere, its value should be a language tag as defined in 6.1. Language Identification.

This element is intended for use only where no other element is available to mark the phrase or words concerned. The global xml:lang attribute should be used in preference to this element where it is intended to mark the language of the whole of some text element.

The distinct element may be used to identify phrases belonging to sublanguages or registers not generally regarded as true languages.

L'attribut global xml:lang doit être fourni dans cet élément pour identifier la langue à laquelle appartiennent le mot ou l'expression balisée. Comme ailleurs, sa valeur devrait être une balise de langue définie dans 6.1. Language Identification.

Cet élément n'est utilisé que lorsqu'il n'y a pas d'autre élément disponible pour baliser l'expression ou les mots concernés. L'attribut global xml:lang doit être préféré à cet élément lorsqu'il s'agit de caractériser la langue de l'ensemble d'un élément textuel.

L'élément distinct peut être utilisé pour identifier des expressions appartenant à des variétés de langue ou à des registres qui ne sont pas généralement considérés comme de vraies langues.

El atributo global xml:lang se debe suministrar para que este elemento identifique la lengua de la palabra o de la frase marcada. Como en otras partes, su valor debe ser una etiqueta de lengua según lo definido en 6.1. Language Identification.

Este elemento se reserva para los casos en que no hay disponible otro elemento para marcar la frase o las palabras referidas. Es preferente el uso del atributo global xml:lang a este elemento donde se quiere marcar la lengua del conjunto de un cierto elemento del texto.

El elemento distinto se puede utilizar para identificar las frases que pertenecen a sublenguajes o a registros no considerados generalmente como lenguas verdaderas.

マークアップされた語句の言語を特定するには、グローバル属性 xml:langを使うべきである。その値は6.1. Language Identificationで定義されている言語コードであるべきである。

当該要素は、当該語句に他の要素が使えない時にのみ使用されることが期待されている。グローバル属性xml:langが、テキスト全体の言語を示すために使用されている場合でも、当該要素の言語を指示するために使用されるべきである。

要素distinctは、特殊言語、すなわち正統な言語とは一般には認められていない言語による語句を示す場合にも使われるかもしれない。

Das globale xml:lang-Attribut sollte mit diesem Element verwendet werden, um die Sprache des markierten Wortes oder der markierten Phrase anzugeben. Der Wert dieses Attributs soll den Empfehlungen von 6.1. Language Identification folgen.

Das foreign-Element sollte nur dann benutzt werden, wenn sonst keine anderen Elemente zur Verfügung stehen, um das betroffene Wort oder die Phrase zu markieren. Wird das foreign-Element nicht verwendet, sollte das globale xml:lang-Attribut bevorzugt verwendet werden, um eine Sprache dem Inhalt eines Elements zuzuweisen.

Das distinct-Element kann verwendet werden, um Phrasen, die zu Subsprachen, Sprachregister oder Varietäten gehören, auszuzeichnen.

3.3.2.1. Foreign Words or Expressions ❌ foreign ❌ 외국어 ❌ 外文 ❌ étranger ❌ extranjero ❌ straniero ❌ fremd ❌ identifies a word or phrase as belonging to some language other than that of the surrounding text. ❌ 주변 텍스트와 다른 언어로 사용된 단어나 구를 표시한다. ❌ 標明所屬語言不同於週遭文字的字詞。 ❌ 記述のベースで使われている言語と異なる言語による語句を定義する。 ❌ reconnaît un mot ou une expression comme appartenant à une langue différente de celle du contexte. ❌ identidica una palabra o frase perteneciente a otra lengua distinta de la del resto del texto en que se incluye. ❌ identifica una parola o un sintagma appartenente ad una lingua diversa da quella del testo circostante. ❌ identifiziert ein Wort oder eine Phrase, die zu einer anderen Sprache gehört, als der umgebende Text. ❌ ❌ ❌ ❌ ❌ ❌ ❌ ❌ ❌ ❌ ❌ ❌ lexeme ❌ Treats the quote as a lexeme ❌ ❌ ❌ ❌ ❌ ❌ This is heathen Greek to you still? Your lapis philosophicus? ❌ ❌ ❌ Et vobis messieurs, Ce ne seroyt que bon que nous rendissiez noz cloches... ❌ ❌ ❌ ❌

Pendant ce temps-là, dans le bureau du rez- de-chaussée, les secrétaires faisaient du hulla-hoop.

❌ ❌ ❌ ❌ 整天窩在家裏看卡通打電動，你是御宅族嗎? ❌ ❌ ❌

The global xml:lang attribute should be supplied for this element to identify the language of the word or phrase marked. As elsewhere, its value should be a language tag as defined in 6.1. Language Identification.

❌

This element is intended for use only where no other element is available to mark the phrase or words concerned. The global xml:lang attribute should be used in preference to this element where it is intended to mark the language of the whole of some text element.

❌

The distinct element may be used to identify phrases belonging to sublanguages or registers not generally regarded as true languages.

❌ ❌ ❌

L'attribut global xml:lang doit être fourni dans cet élément pour identifier la langue à laquelle appartiennent le mot ou l'expression balisée. Comme ailleurs, sa valeur devrait être une balise de langue définie dans 6.1. Language Identification.

❌

Cet élément n'est utilisé que lorsqu'il n'y a pas d'autre élément disponible pour baliser l'expression ou les mots concernés. L'attribut global ❌ xml:lang doit être préféré à cet élément lorsqu'il s'agit de caractériser la langue de l'ensemble d'un élément textuel.

❌

L'élément distinct peut être utilisé pour identifier des expressions appartenant à des variétés de langue ou à des registres qui ne sont pas généralement considérés comme de vraies langues.

❌ ❌ ❌

El atributo global xml:lang se debe suministrar para que este elemento identifique la lengua de la palabra o de la frase marcada. Como en otras partes, su valor debe ser una etiqueta de lengua según lo definido en 6.1. Language Identification.

❌

Este elemento se reserva para los casos en que no hay disponible otro elemento para marcar la frase o las palabras referidas. Es preferente el uso del atributo global xml:lang a este elemento donde se quiere marcar la lengua del conjunto de un cierto elemento del texto.

❌

El elemento distinto se puede utilizar para identificar las frases que pertenecen a sublenguajes o a registros no considerados generalmente como lenguas verdaderas.

❌ ❌ ❌

マークアップされた語句の言語を特定するには、グローバル属性 xml:langを使うべきである。その値は6.1. Language Identificationで定義されている言語コードであるべきである。

❌

当該要素は、当該語句に他の要素が使えない時にのみ使用されることが期待されている。グローバル属性xml:langが、テキスト全体の言語を示すために使用されている場合でも、当該要素の言語を指示するために使用されるべきである。

❌

要素distinctは、特殊言語、すなわち正統な言語とは一般には認められていない言語による語句を示す場合にも使われるかもしれない。

❌ ❌ ❌

Das globale xml:lang-Attribut sollte mit diesem Element verwendet werden, um die Sprache des markierten Wortes oder der markierten Phrase anzugeben. Der Wert dieses Attributs soll den Empfehlungen von 6.1. Language Identification folgen.

❌

Das foreign-Element sollte nur dann benutzt werden, wenn sonst keine anderen Elemente zur Verfügung stehen, um das betroffene Wort oder die Phrase zu markieren. Wird das foreign-Element nicht verwendet, sollte das globale xml:lang-Attribut bevorzugt verwendet werden, um eine Sprache dem Inhalt eines Elements zuzuweisen.

❌

Das distinct-Element kann verwendet werden, um Phrasen, die zu Subsprachen, Sprachregister oder Varietäten gehören, auszuzeichnen.

❌ ❌ ❌ 3.3.2.1. Foreign Words or Expressions ❌ ```

Merging the two attDef elements in a single elementSpec/attList results in a valid intermediary ODD and RNG:

        <elementSpec ident="foreign" module="core" mode="change">
          <attList>
            <attDef ident="xml:lang" usage="req" mode="change">
              <valList type="closed" mode="add">
                <valItem ident="ang">
                  <gloss>Anglo-Saxon</gloss>
                  <desc>Anglo-Saxon</desc>
                </valItem>
              </valList>
            </attDef>
            <attDef ident="ana" usage="opt" mode="change">
              <valList type="closed" mode="add">
                <valItem ident="lexeme">
                  <gloss>lexeme</gloss>
                  <desc>Treats the quote as a lexeme</desc>
                </valItem>
              </valList>
            </attDef>
          </attList>
        </elementSpec>

I'd suggest having a look at (recent changes of) the rules/scopes in odd2odd.xsl.

joeytakeda commented 2 months ago

Thanks @nivaca for this detailed report! Really helpful for tracking down this bug.

I've compared the results from 7.55, 7.56, and 7.56a and can confirm that this is an issue introduced between 7.55 and 7.56 with respect to elementSpecs with the same @ident (see also #645). Per @raffazizzi 's comment there, "this is a bug and not intentional" (https://github.com/TEIC/Stylesheets/issues/645#issuecomment-1840947279). @sydb also notes the fix in #645 "does not fix the duplicate attrs defined in Relax NG problem, which appears to have been introduced quite recently, and should be a separate ticket."

I think this is high priority, and should be fixed ASAP. From my testing, it appears that issue emerged from the changes to the tei:uniqueName function — reverting that function to its previous state in 7.55 fixes the issue.

Running the current dev Stylesheets yields the following schemaSpec as the results of pass0 (e.g: $ODD) — here the two duplicate elementSpecs are retained:

<schemaSpec xmlns:teix="http://www.tei-c.org/ns/Examples" ident="SMTrad" source="https://www.tei-c.org/Vault/P5/current/xml/tei/odd/p5subset.xml" defaultExceptions="http://www.tei-c.org/ns/1.0 teix:egXML">

            <moduleRef key="core"/>
            <moduleRef key="tei"/>
            <moduleRef key="header"/>

            <elementSpec ident="foreign" module="core" mode="change">
               <attList>
                  <attDef ident="xml:lang" usage="req" mode="change">
                     <valList type="closed" mode="add">
                        <valItem ident="ang">
                           <gloss>Anglo-Saxon</gloss>
                           <desc>Anglo-Saxon</desc>
                        </valItem>
                     </valList>
                  </attDef>
               </attList>
            </elementSpec>

            <elementSpec ident="foreign" module="core" mode="change">
               <attList>
                  <attDef ident="ana" usage="opt" mode="change">
                     <valList type="closed" mode="add">
                        <valItem ident="lexeme">
                           <gloss>lexeme</gloss>
                           <desc>Treats the quote as a lexeme</desc>
                        </valItem>
                     </valList>
                  </attDef>
               </attList>
            </elementSpec>

         </schemaSpec>

Running the current stylesheets with the 7.55 version of tei:uniqueName merges the two elementSpecs into a single one with two attLists instead:

         <schemaSpec xmlns:teix="http://www.tei-c.org/ns/Examples" ident="SMTrad" source="https://www.tei-c.org/Vault/P5/current/xml/tei/odd/p5subset.xml" defaultExceptions="http://www.tei-c.org/ns/1.0 teix:egXML">

            <moduleRef key="core"/>
            <moduleRef key="tei"/>
            <moduleRef key="header"/>

            <elementSpec ident="foreign" module="core" mode="change">
               <attList>
                  <attDef ident="xml:lang" usage="req" mode="change">
                     <valList type="closed" mode="add">
                        <valItem ident="ang">
                           <gloss>Anglo-Saxon</gloss>
                           <desc>Anglo-Saxon</desc>
                        </valItem>
                     </valList>
                  </attDef>
               </attList>

               <attList>
                  <attDef ident="ana" usage="opt" mode="change">
                     <valList type="closed" mode="add">
                        <valItem ident="lexeme">
                           <gloss>lexeme</gloss>
                           <desc>Treats the quote as a lexeme</desc>
                        </valItem>
                     </valList>
                  </attDef>
               </attList>
            </elementSpec>
         </schemaSpec>

My best guess so far is that the problem is in the initial processing of *Spec/@mode='change' : https://github.com/TEIC/Stylesheets/blob/5a48e1595f0bf7d35c1dcd50993724cd346d4cd9/odds/odd2odd.xsl#L510-L532

In particular, I think it's these lines:

https://github.com/TEIC/Stylesheets/blob/5a48e1595f0bf7d35c1dcd50993724cd346d4cd9/odds/odd2odd.xsl#L512-L514

Where this could be true with the old uniqueName function (since TEI elements were identified by their local name — e.g. in this case, "foreign"), idents and uniqueNames are going to be different. Working on this in branch iss678_duplicateIdents

joeytakeda commented 2 months ago

I've made this change in commit https://github.com/TEIC/Stylesheets/commit/8aefd67c482fea69e6bb8fcd5966f4329cd5a1b2 , which does produce the correct processed.odd and RNG from the sample ODD. However, tests are failing, so that requires further investigation

.rng

```xml (foreign) identifies a word or phrase as belonging to some language other than that of the surrounding text. [3.3.2.1. Foreign Words or Expressions] (language) indicates the language of the element content using a tag generated according to BCP 47. ang (Anglo-Saxon) Anglo-Saxon ```

.processed.odd

```xml foreign 외국어 外文 étranger extranjero straniero fremd identifies a word or phrase as belonging to some language other than that of the surrounding text. 주변 텍스트와 다른 언어로 사용된 단어나 구를 표시한다. 標明所屬語言不同於週遭文字的字詞。記述のベースで使われている言語と異なる言語による語句を定義する。 reconnaît un mot ou une expression comme appartenant à une langue différente de celle du contexte. identidica una palabra o frase perteneciente a otra lengua distinta de la del resto del texto en que se incluye. identifica una parola o un sintagma appartenente ad una lingua diversa da quella del testo circostante. identifiziert ein Wort oder eine Phrase, die zu einer anderen Sprache gehört, als der umgebende Text. Anglo-Saxon Anglo-Saxon lexeme Treats the quote as a lexeme This is heathen Greek to you still? Your lapis philosophicus? Et vobis messieurs, Ce ne seroyt que bon que nous rendissiez noz cloches...