search

relaton / relaton-bipm

MIT License
2 stars 0 forks source link

Support French retrieval of Resolution, Recommendation, Declaration and Meeting #27

Closed ronaldtse closed 2 years ago

ronaldtse commented 2 years ago

The fetching syntax should allow both English and French, and retrieve the identical bibliographic item. This is because BIPM is a French-first organization.

In rendering the title/contents, the language should respect the rendering options.

Originally from @manuel489 (https://github.com/metanorma/bipm-si-brochure/issues/169):

French references don't fetch when a tilde is present in the identifier

andrew2net commented 2 years ago

@ronaldtse it was implemented in v 1.11.6

ronaldtse commented 2 years ago

Thanks @andrew2net !

manuelfuenmayor commented 2 years ago

@andrew2net @ronaldtse

I'm compiling with relaton-bipm v 1.11.8 and these references are not fetching:

capture4

manuelfuenmayor commented 2 years ago

Could anyone else confirm my outcome, please?

andrew2net commented 2 years ago

@manuel489 for me relaton finds these references:

$ relaton fetch 'CGPM Résolution 1889-00'     
[relaton-bipm] ("CGPM Résolution 1889-00") fetching...
[relaton-bipm] ("CGPM Résolution 1889-00") found CGPM Resolution (1889)
<bibitem id="CGPMResolution(1889)">
...

$ relaton fetch 'CGPM Résolution 1901-01'
[relaton-bipm] ("CGPM Résolution 1901-01") fetching...
[relaton-bipm] ("CGPM Résolution 1901-01") found CGPM Resolution 1 (1901)
<bibitem id="CGPMResolution1(1901)">
...

$ relaton fetch 'CGPM Résolution 1901-02'
[relaton-bipm] ("CGPM Résolution 1901-02") fetching...
[relaton-bipm] ("CGPM Résolution 1901-02") found CGPM Resolution 2 (1901)
<bibitem id="CGPMResolution2(1901)">
...

$ relaton fetch 'CGPM Résolution 1927-01'
[relaton-bipm] ("CGPM Résolution 1927-01") fetching...
[relaton-bipm] ("CGPM Résolution 1927-01") found CGPM Resolution 1 (1927)
<bibitem id="CGPMResolution1(1927)">
...

$ relaton fetch 'CGPM Réunion 9'         
[relaton-bipm] ("CGPM Réunion 9") fetching...
[relaton-bipm] ("CGPM Réunion 9") found CGPM Meeting 9
<bibitem id="CGPMMeeting9">
...

$ relaton fetch 'CGPM Résolution 1948-01'
[relaton-bipm] ("CGPM Résolution 1948-01") fetching...
[relaton-bipm] ("CGPM Résolution 1948-01") found CGPM Resolution 1 (1948)
<bibitem id="CGPMResolution1(1948)">
...

$ relaton fetch 'CGPM Résolution 1948-02'
[relaton-bipm] ("CGPM Résolution 1948-02") fetching...
[relaton-bipm] ("CGPM Résolution 1948-02") found CGPM Resolution 2 (1948)
<bibitem id="CGPMResolution2(1948)">
...
anermina commented 2 years ago

@andrew2net which relaton-bipm version are you using? I am currently getting the same output as Manuel.

manuelfuenmayor commented 2 years ago

Maybe this is a metanorma-bipm issue, I have opened a ticket for @opoudjis there.

andrew2net commented 2 years ago

which relaton-bipm version are you using?

@anermina I use v 1.11.8

opoudjis commented 2 years ago

First:

relaton fetch is NOT a useful test of this, because the functionality is invoked in fetch_async.

I do not know what is happening, but under Metanorma operating conditions, which means caching and asynchronous fetching, "CGPM Résolution 1889-00" is NOT being recognised at all. And @andrew2net needs to debug this in the context of fetch_async. I can't.

Second:

What is required here by @ronaldtse refusing to accept i18n of the identifier after fetching is a bilingual identifier. That is not being fetched at all.

bundle exec relaton fetch "CGPM Résolution 1889-00"

is fetching:

<bibitem id="CGPMResolution(1889)">
  <fetched>2022-06-13</fetched>
  <title format="text/plain" language="en" script="Latn">Sanction of the international prototypes of the metre and the kilogram</title>
  <title format="text/plain" language="fr" script="Latn">Sanction des prototypes internationaux du mètre et du kilogramme</title>
  <uri type="src">https://www.bipm.org/en/committees/cg/cgpm/1-1889/resolution-</uri>
  <uri type="src">https://raw.githubusercontent.com/metanorma/bipm-data-outcomes/main/cgpm/meetings-en/meeting-01.yml</uri>
  <uri type="src">https://raw.githubusercontent.com/metanorma/bipm-data-outcomes/main/cgpm/meetings-fr/meeting-01.yml</uri>
  <uri type="pdf">https://www.bipm.org/documents/20126/17314988/CGPM1.pdf/ab69792a-2e5e-2040-14d0-11464bc9414c</uri>
  <docidentifier type="BIPM" primary="true">CGPM Resolution (1889)</docidentifier>
  <docnumber>CGPM Resolution (1889)</docnumber>
  <date type="published">
    <on>1889-09-28</on>
  </date>
  <contributor>
    <role type="publisher"/>
    <organization>
      <name>Bureau International des Poids et Mesures</name>
      <abbreviation>BIPM</abbreviation>
      <uri>www.bipm.org</uri>
    </organization>
  </contributor>
  <language>en</language>
  <language>fr</language>
  <script>Latn</script>
</bibitem>

I need there to be a

CGPM Résolution (1889)

somewhere in the record. Right now, there is not.

Do NOT give me different output if I request "CGPM Résolution 1889-00" or "CGPM Resolution 1889-00". Manuel and Ronald seem to assume you should: that is completely wrong. There is only one bibliographic resource, with language variants inside.

Instead, give me three identifiers:

<docidentifier type="BIPM" primary="true">CGPM Resolution (1889)</docidentifier>
<docidentifier type="BIPM" primary="true" language="en">CGPM Resolution (1889)</docidentifier>
<docidentifier type="BIPM" primary="true" language="fr">CGPM Résolution (1889)</docidentifier>

The notion of a language-specific document identifier is insane, but... I don't think it's wrong. I'll add language and script to the grammar for docidentifier.

@andrew2net I need you to test this on branch fix/relaton-i18n of metanorma-bipm, so ensure that CGPM Resolution (1889) is being fetched; right now:

        [bibliography]
        == Bibliography
        * [[[a1,CGPM Résolution 1889-00]]]
        * [[[a2,CIPM Décision 2016-01]]]

is not being recognised, and is being passed through as unrecognised:

         <bibliography>
           <references id='_' normative='false' obligation='informative'>
             <title>Bibliography</title>
             <bibitem id='a1'>
               <formattedref format='application/x-isodoc+xml'/>
               <docidentifier type='BIPM'>CGPM Résolution 1889-00</docidentifier>
               <docnumber>1889-00</docnumber>
             </bibitem>
             <bibitem id='a2'>
               <formattedref format='application/x-isodoc+xml'/>
               <docidentifier type='BIPM'>CIPM Décision 2016-01</docidentifier>
               <docnumber>2016-01</docnumber>
             </bibitem>
           </references>
         </bibliography>
       </bipm-standard>
ronaldtse commented 2 years ago

Do NOT give me different output if I request "CGPM Résolution 1889-00" or "CGPM Resolution 1889-00". Manuel and Ronald seem to assume you should: that is completely wrong. There is only one bibliographic resource, with language variants inside.

You misread what I meant.

See above:

The fetching syntax should allow both English and French, and retrieve the identical bibliographic item.

The SAME item should be returned regardless of fetching syntax.

ronaldtse commented 2 years ago

@andrew2net can you please help here?

1.

"CGPM Résolution 1889-00" is NOT being recognised at all.

2.

bundle exec relaton fetch "CGPM Résolution 1889-00"

This needs to return the docidentifiers of all languages.

opoudjis commented 2 years ago

"CGPM Résolution 1889-00" is NOT being recognised at all.

Fixed; thank you for finding the issue, @andrew2net

andrew2net commented 2 years ago

@ronaldtse can we close this issue?

opoudjis commented 2 years ago

No, because we need:

  <docidentifier type="BIPM" primary="true">CGPM Resolution (1889)</docidentifier>
  <docidentifier type="BIPM" primary="true" language="en" script="Latn">CGPM Resolution (1889)</docidentifier>
  <docidentifier type="BIPM" primary="true" language="fr" script="Latn">CGPM Résolution (1889)</docidentifier>
ronaldtse commented 2 years ago

FYI @andrew2net is currently working on this.

opoudjis commented 2 years ago

I need the docidentifier instances differentiated by language attributes. This has not been done:

bundle exec relaton fetch "CGPM Meeting 1"

<bibitem id="CGPMMeeting1">
  <fetched>2022-07-02</fetched>
  <title format="text/plain" language="en" script="Latn">1st meeting of the CGPM</title>
  <title format="text/plain" language="fr" script="Latn">1e réunion de la CGPM</title>
  <uri type="src">https://www.bipm.org/en/committees/cg/cgpm/1-1889</uri>
  <uri type="pdf">https://www.bipm.org/documents/20126/38097196/CGPM1.pdf/0ff415d6-87a2-5e23-5a1c-cadf8b8047a6</uri>
  <uri type="src">https://raw.githubusercontent.com/metanorma/bipm-data-outcomes/main/cgpm/meetings-en/meeting-01.yml</uri>
  <uri type="src">https://raw.githubusercontent.com/metanorma/bipm-data-outcomes/main/cgpm/meetings-fr/meeting-01.yml</uri>
  <docidentifier type="BIPM" primary="true">CGPM Meeting 1</docidentifier>
  <docidentifier type="BIPM" primary="true">CGPM Meeting 1</docidentifier>
  <docidentifier type="BIPM" primary="true">CGPM Réunion 1</docidentifier>
  <docnumber>CGPM Meeting 1</docnumber>
  <date type="published">
    <on>1889-09-28</on>
  </date>
  <contributor>
    <role type="publisher"/>
    <organization>
      <name>Bureau International des Poids et Mesures</name>
      <abbreviation>BIPM</abbreviation>
      <uri>www.bipm.org</uri>
    </organization>
  </contributor>
  <language>en</language>
  <language>fr</language>
  <script>Latn</script>
</bibitem>

What I need is:

  <docidentifier type="BIPM" primary="true">CGPM Meeting 1</docidentifier>
  <docidentifier type="BIPM" primary="true" language="en">CGPM Meeting 1</docidentifier>
  <docidentifier type="BIPM" primary="true" language="fr">CGPM Réunion 1</docidentifier>
andrew2net commented 2 years ago

@opoudjis I'm unable reproduce the issue. I get:

relaton fetch "CGPM Meeting 1"
[relaton-bipm] ("CGPM Meeting 1") fetching...
[relaton-bipm] ("CGPM Meeting 1") found CGPM Meeting 1
<bibitem id="CGPMMeeting1">
  <fetched>2022-07-02</fetched>
  <title format="text/plain" language="en" script="Latn">1st meeting of the CGPM</title>
  <title format="text/plain" language="fr" script="Latn">1e réunion de la CGPM</title>
  <uri type="src">https://www.bipm.org/en/committees/cg/cgpm/1-1889</uri>
  <uri type="pdf">https://www.bipm.org/documents/20126/38097196/CGPM1.pdf/0ff415d6-87a2-5e23-5a1c-cadf8b8047a6</uri>
  <uri type="src">https://raw.githubusercontent.com/metanorma/bipm-data-outcomes/main/cgpm/meetings-en/meeting-01.yml</uri>
  <uri type="src">https://raw.githubusercontent.com/metanorma/bipm-data-outcomes/main/cgpm/meetings-fr/meeting-01.yml</uri>
  <docidentifier type="BIPM" primary="true">CGPM Meeting 1</docidentifier>
  <docidentifier type="BIPM" primary="true" language="en" script="Latn">CGPM Meeting 1</docidentifier>
  <docidentifier type="BIPM" primary="true" language="fr" script="Latn">CGPM Réunion 1</docidentifier>
  <docnumber>CGPM Meeting 1</docnumber>
  <date type="published">
    <on>1889-09-28</on>
  </date>
  <contributor>
    <role type="publisher"/>
    <organization>
      <name>Bureau International des Poids et Mesures</name>
      <abbreviation>BIPM</abbreviation>
      <uri>www.bipm.org</uri>
    </organization>
  </contributor>
  <language>en</language>
  <language>fr</language>
  <script>Latn</script>
</bibitem>
ronaldtse commented 2 years ago

@andrew2net I'm on relaton-bipm 1.11.9 but seeing the same output as @opoudjis ?

opoudjis commented 2 years ago

right, needs relaton version update from standoc

andrew2net commented 2 years ago

@andrew2net I'm on relaton-bipm 1.11.9 but seeing the same output as @opoudjis ?

@ronaldtse because the Relaton model is updated in the version 1.12

ronaldtse commented 2 years ago

@opoudjis relaton-bipm is still locked to 1.11.9.

opoudjis commented 2 years ago

Yes, release is tonight.