Block issues in latest auto-generated drafts

[jerstlouis]

In the latest auto-generated drafts, blocks are not being rendered properly, being all on one line e.g.:

from https://docs.ogc.org/DRAFTS/20-058.html#_link_from_data_resource

whereas with my local fairly recent metanorma build this does not occur:

Metanorma 2.0.1 Metanorma::Cli 1.10.1 Metanorma::Ogc 2.5.10

There are several occurences of this throughout OGC API - Maps.

not sure what is causing this...

cc. @gbuehler @ghobona @opoudjis

[opoudjis]

I suspect it's https://github.com/metanorma/metanorma-utils/releases/tag/v1.9.0, updates not to treat carriage return as whitespace for Chinese/Japanese/Korean text (we're working with Japanese now), and allowing fewer carriage returns in the XML than we used to as a result. Source code was expressly meant not to be impacted by this; I will need to investigate to see what's happened here, but I've got hold of this too late for today's release.

[opoudjis]

Ugh.

Sourcecode is preserving carriage return fine. But sourcecode wrapped in examples, as with that example, is now not.

[opoudjis]

This is going to be difficult. I have to re-conceive how Nokogiri is processed with regard to the line endings of blocks: they cannot be handled generically in metanorma-utils noko(), but instead need to be handled on a block-by-block basis (as they already are supposed to be), except that nested blocks need to be left alone as single concatenated list elements.

[opoudjis]

Yeah, the solution to this is going to have to be:

• Do not do any line-stripping (replacing of carriage return with space, including CJK detection), in noko()
• Do the line stripping in metanorma-standoc in text postprocessing, in the same context as smartquotes_cleanup(xmldoc).
• Just as smartquotes_cleanup(), the line stripping will ignore any text whose ancestor is passthrough text (and would preserve dumb quotes): carriage returns are left alone in text descended from sourcecode, pre, passthrough, etc, but they are stripped in all other spans of text in Metanorma.

Release has been held up because of fontist issues, so I will fix this tomorrow and re-release.

[opoudjis]

Will test entire Metanorma stack and re-release tomorrow, but these fixes address this issue. There will be a bit fewer carriage returns than before in the XML; the condition for removing a carriage return is that it is inside a block and not a preformatted block. So examples like this, with example/p and example/sourcedoc, will remove the carriage return between the example/p and the example/sourcedoc, and within example/p, but not within example/sourcedoc:

<example><p>A B C</p> <sourcedoc>{
  A
  B
}</sourcecode> </example>

@CAMOBAP Release is on hold anyway, but I will be incrementing standoc release.

[opoudjis]

Still need to write test for this new code in standoc, and test remainder of Metanorma.

[jerstlouis]

@opoudjis There was also several occurences of this well for the OpenAPI snippets inside of the requirements -- I'm not sure whether it is the exact same scenario or not (also happening in that OGC API - Maps document).

[opoudjis]

Realising that, just as with smart quotes, we need to be aware of next line of text within block:

<p>
A
<em>B</em>
C
</p>

B will be traversed separately from A and C by Nokogiri, since we are isolating text spans; B needs to know that it is followed by C, and whether C is in CJK or not. This needs to be fixed, because right now space is being removed before inline markup.

[opoudjis]

After a lot of to-and-fro, I'm satisfied I've addressed this; attaching latest ogc api maps. Will do updated release late tonight, although it is still being held up by https://github.com/fontist/fontist/issues/382

20-058.html.zip

[ghobona]

@jerstlouis Thanks for spotting the issue. We will re-check before publishing 20-058.

Cc: @gbuehler