anima-wg / anima-brski-ae

BRSKI with alternative enrollments
Other
4 stars 1 forks source link

Cannot use SVG image due to xml2rfc errors #22

Closed DDvO closed 11 months ago

DDvO commented 2 years ago

As suggested by @toerless on Friday, I tried adding an image extracted from the presentation, but cannot recall how I was supposed to do this.

When I add it as an image to the MarkDown source file like this:

![BRSKI-AE abstract protocol overview](BRSKI-AE_abstract_protocol_overview.png "BRSKI-AE abstract protocol overview")

then

 kdrfc --v3 draft-ietf-anima-brski-async-enroll.md

fails like this:

draft-ietf-anima-brski-async-enroll.xml(595): Error: Did not expect element xref there, at /rfc/middle/section[4]/section[2]/xref
draft-ietf-anima-brski-async-enroll.xml(595): Error: Element section has extra content: xref, at /rfc/middle/section[4]/section[2]/xref
/home/david/Siemens/proj/PKI/anima-brski-async-enroll-internal/draft-ietf-anima-brski-async-enroll.xml(15): Error: Invalid document before running preptool.
Unable to complete processing draft-ietf-anima-brski-async-enroll.xml

Does it make sense in such a way after all, or how else to add the image?

toerless commented 2 years ago

Hi David

I have actually never tried to insert graphics into draft/RFCs, mostly because in the recent ones i didn't have any such good graphical material. And i think its a widely underused/underdocumented option. And maybe its even over-regulated, aka: in the end some process will tell us we can not do it as we may like to do it.

Here is an example you may want to clone:

https://datatracker.ietf.org/doc/html/rfc7893

It has in its PDF nice colorful graphics, and in their place in the txt and any other rendering only placeholder blocks. And it has some text pointing readers to use the PDF rendering to see the graphics. This is the type of example RFC from which i would clone the formatting for such PDF-only pictures (maybe there are newer RFCs using the same tricks).

You can see the XML formatting used in the RFCs draft XML file:

https://www.ietf.org/archive/id/draft-ietf-pals-congcons-02.xml

Now, i do not know which tool to use to get the PDF rendering, and i also do not know if/how it would be possible to have a markdown file that does create the desired XML, but:

How about you first try to hack in the XML required by hand and see if you can generate a PDF with the image inlined - because i am pretty sure that you would need to upload both the xml and the pre-rendered PDF to datatracker if you wanted the PDF correctly rendered on data tracker

Once you know that step, we can ask a markdown expert whether or how the necessary XML can be created from markdown.

If any of these tooling steps turns out to be too cumbersome for the draft versions, I would suggest you just create the placeholder ascii-art in markdown and fill it with:

a) a URL to a github location of the png image b) [RFC-editor please insert png image into PDF rendering of RFC] instructions for RFC editor.

Aka: Don't bother with a complex tool-chain during writing of the draft versions, but push it down the road to the time when the RFC is being built, and just have the right text and URL pointer in the document to make sure this happens once the doc goes RFC.

Only downside is that readers of the draft will have to click on the URL and then can only see the picture in a separate browser window (big deal ;-).

Cheers Toerless

On Mon, Mar 28, 2022 at 03:00:21AM -0700, David von Oheimb wrote:

As suggested by @toerless on Friday, I tried adding an image extracted from the presentation, but cannot recall how I was supposed to do this.

When I add it as an image to the MarkDown source file like this:

![BRSKI-AE abstract protocol overview](BRSKI-AE_abstract_protocol_overview.png "BRSKI-AE abstract protocol overview")

then

 kdrfc --v3 draft-ietf-anima-brski-async-enroll.md

fails like this:

draft-ietf-anima-brski-async-enroll.xml(595): Error: Did not expect element xref there, at /rfc/middle/section[4]/section[2]/xref
draft-ietf-anima-brski-async-enroll.xml(595): Error: Element section has extra content: xref, at /rfc/middle/section[4]/section[2]/xref
/home/david/Siemens/proj/PKI/anima-brski-async-enroll-internal/draft-ietf-anima-brski-async-enroll.xml(15): Error: Invalid document before running preptool.
Unable to complete processing draft-ietf-anima-brski-async-enroll.xml

Does it make sense in such a way after all, or how else to add the image?

-- Reply to this email directly or view it on GitHub: https://github.com/anima-wg/anima-brski-async-enroll/issues/22 You are receiving this because you were mentioned.

Message ID: @.***>

--

@.***

mcr commented 2 years ago

It's really not worth trying this. Just reference the slides as an informational refernece.

HBrock commented 2 years ago

Hi David

Andreas Kretschmer experienced a little with embedding UML into XML based I-Ds. The formatting of the supported UML seamed to be poor. Therefore, we decided to use ACSII-art again. Maybe ASCII-art also works for your flow chart.

Hendrik

toerless commented 2 years ago

Btw: I am all for ASCII-art when it fits. And i think it can even be combined with the full png art in the PDF: Just replace the ASCII placeholder art with some "as-good-as-possible" ASCII art, but still do the full png in the PDF and point that out in the text.

But then again, i am one of those folks who spent a lot of effort on IMHO very useful slides, and regrets that none of them ended up being published in long-term documents like RFCs (such as rfc8994/8995 - because it was too late in the process to suggest), so don't listen to me if you didn't waste lifetime on such efforts ;-))

Cheers Toerless

On Mon, Mar 28, 2022 at 11:17:05PM -0700, Hendrik Brockhaus wrote:

Hi David

Andreas Kretschmer experienced a little with embedding UML into XML based I-Ds. The formatting of the supported UML seamed to be poor. Therefore, we decided to use ACSII-art again. Maybe ASCII-art also works for your flow chart.

Hendrik

Von: Toerless Eckert @.> Gesendet: Montag, 28. März 2022 17:48 An: anima-wg/anima-brski-async-enroll @.> Cc: Subscribed @.***> Betreff: Re: [anima-wg/anima-brski-async-enroll] Cannot insert image in markdown source (Issue #22)

Hi David

I have actually never tried to insert graphics into draft/RFCs, mostly because in the recent ones i didn't have any such good graphical material. And i think its a widely underused/underdocumented option. And maybe its even over-regulated, aka: in the end some process will tell us we can not do it as we may like to do it.

Here is an example you may want to clone:

https://datatracker.ietf.org/doc/html/rfc7893

It has in its PDF nice colorful graphics, and in their place in the txt and any other rendering only placeholder blocks. And it has some text pointing readers to use the PDF rendering to see the graphics. This is the type of example RFC from which i would clone the formatting for such PDF-only pictures (maybe there are newer RFCs using the same tricks).

You can see the XML formatting used in the RFCs draft XML file:

https://www.ietf.org/archive/id/draft-ietf-pals-congcons-02.xml

Now, i do not know which tool to use to get the PDF rendering, and i also do not know if/how it would be possible to have a markdown file that does create the desired XML, but:

How about you first try to hack in the XML required by hand and see if you can generate a PDF with the image inlined - because i am pretty sure that you would need to upload both the xml and the pre-rendered PDF to datatracker if you wanted the PDF correctly rendered on data tracker

Once you know that step, we can ask a markdown expert whether or how the necessary XML can be created from markdown.

If any of these tooling steps turns out to be too cumbersome for the draft versions, I would suggest you just create the placeholder ascii-art in markdown and fill it with:

a) a URL to a github location of the png image b) [RFC-editor please insert png image into PDF rendering of RFC] instructions for RFC editor.

Aka: Don't bother with a complex tool-chain during writing of the draft versions, but push it down the road to the time when the RFC is being built, and just have the right text and URL pointer in the document to make sure this happens once the doc goes RFC.

Only downside is that readers of the draft will have to click on the URL and then can only see the picture in a separate browser window (big deal ;-).

Cheers Toerless

On Mon, Mar 28, 2022 at 03:00:21AM -0700, David von Oheimb wrote:

As suggested by @toerless on Friday, I tried adding an image extracted from the presentation, but cannot recall how I was supposed to do this.

When I add it as an image to the MarkDown source file like this:

![BRSKI-AE abstract protocol overview](BRSKI-AE_abstract_protocol_overview.png "BRSKI-AE abstract protocol overview")

then

kdrfc --v3 draft-ietf-anima-brski-async-enroll.md

fails like this:

draft-ietf-anima-brski-async-enroll.xml(595): Error: Did not expect element xref there, at /rfc/middle/section[4]/section[2]/xref
draft-ietf-anima-brski-async-enroll.xml(595): Error: Element section has extra content: xref, at /rfc/middle/section[4]/section[2]/xref
/home/david/Siemens/proj/PKI/anima-brski-async-enroll-internal/draft-ietf-anima-brski-async-enroll.xml(15): Error: Invalid document before running preptool.
Unable to complete processing draft-ietf-anima-brski-async-enroll.xml

Does it make sense in such a way after all, or how else to add the image?

-- Reply to this email directly or view it on GitHub: https://github.com/anima-wg/anima-brski-async-enroll/issues/22 You are receiving this because you were mentioned.

Message ID: @.<mailto:@.>>

--

@.<mailto:@.>

- Reply to this email directly, view it on GitHubhttps://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fanima-wg%2Fanima-brski-async-enroll%2Fissues%2F22%23issuecomment-1080818366&data=04%7C01%7Chendrik.brockhaus%40siemens.com%7Cc8e5f3d6bff545a49d8c08da10d2521d%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637840792752385911%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=NjGpE4uFYMKsdrG5TDd34cGj6RsCZcyQhwvqzTi5VOQ%3D&reserved=0, or unsubscribehttps://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAIRDLFIX4GYTSC5CRKHBWJDVCHICPANCNFSM5R2TBSSA&data=04%7C01%7Chendrik.brockhaus%40siemens.com%7Cc8e5f3d6bff545a49d8c08da10d2521d%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637840792752385911%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=nKZAZORkA6gUlDbFxdPhTF4r%2B72EDNDA9t30HK9408E%3D&reserved=0. You are receiving this because you are subscribed to this thread.Message ID: @.**@.>>

-- Reply to this email directly or view it on GitHub: https://github.com/anima-wg/anima-brski-ae/issues/22#issuecomment-1081446575 You are receiving this because you were mentioned.

Message ID: @.***>

--

@.***

DDvO commented 2 years ago

Thanks all for your comments and hints.

The support for including graphical images in IETF RFC documents is indeed pretty poor, and documentation around this topic appears even more so.

By some digging and trial&error, I found that

So I ended up including in the master file draft-ietf-anima-brski-ae.md the following XML section:

<figure title="BRSKI-AE abstract protocol overview">
    <!-- Image converted from BRSKI-AE_abstract_protocol_overview.png
         using https://www.freeconvert.com/png-to-svg
         which provides reasonable quality but unfortunately removed some colors.
    -->
    <artset>
        <artwork type="svg" src="BRSKI-AE_abstract_protocol_overview.svg" />
        <artwork>type="ascii-art" src="BRSKI-AE_abstract_protocol_overview.txt"</artwork>
    </artset>
</figure>

referring to the SVG file BRSKI-AE_abstract_protocol_overview.svg and to an auxiilary file BRSKI-AE_abstract_protocol_overview.txt containing

[ Cannot render SVG graphics - please view
  https://raw.githubusercontent.com/anima-wg/anima-brski-ae/main/overview.png ]
toerless commented 2 years ago

David:

Thanks for digging into thr SVG mess, but i think almost everybody hates it.

I'd just stick to getting the png into the PDF as i originally suggested. No artifical limitations on quality or color, minimal work.

Now, if you're interested to complain about the state of affairs with SVG, there are mailing lists for that as well (also likely to no much effect).

Cheers Toerless

DDvO commented 2 years ago

Toerless, thanks for your further response.

I'd just stick to getting the png into the PDF as i originally suggested. No artifical limitations on quality or color, minimal work.

The examples in https://www.ietf.org/archive/id/draft-ietf-pals-congcons-02.xml look like this:

<figure><artwork src="T1E1rvp64.jpg">
<![CDATA[
--------------------------------------------------------------------
I                                                                  I
I             E1/T1 PWs vs. TCP for segment size 64B               I
I                                                                  I
I                     (only in PDF version)                        I
I                                                                  I
--------------------------------------------------------------------
]]>
</artwork>
<postamble> Figure 1  E1/T1 PWs vs. TCP for segment size 64B </postamble>
</figure>

This does give a nice hint how to avoid the trouble and complications I had with <artset> and two <artword> entries. So I improved the XML code to:

<figure title="BRSKI-AE abstract protocol overview">
    <!-- Image converted from PDF using https://www.zamzar.com/convert/pdf-to-svg/-->
    <artwork src="overview.svg">
[ Cannot render SVG graphics - please view
  https://raw.githubusercontent.com/anima-wg/anima-brski-ae/main/overview.png ]
    </artwork>
</figure>

Yet when I try using <artwork> with any binary graphics format (e.g., JPG and PNG), xml2rfc fails with

Error: Discarded unexpected <artwork> content with type='jpg': ''utf-8' codec can't decode byte 0xff in position 0: invalid start byte'
Warning: No image data found in source file overview.jpg

which may due to a bug in current xml2rfc or some Python 3 hick-up.

I worked around this as commented in my new XML snippet, using an SVG file that I meanwhile produce from an intermediate PDF file using https://www.zamzar.com/convert/pdf-to-svg/ This converter copes with a larger number of colors and produces good quality in rather short time.

Using SVG also has the nice merit that HTML output of xml2rfc includes the graphics.

Now, if you're interested to complain about the state of affairs with SVG, there are mailing lists for that as well (also likely to no much effect).

I've just reported an issue at https://github.com/ietf-tools/xml2rfc/issues/751.

toerless commented 2 years ago

Nice!

When i run my local xml2rfc, i can not actually see an option to produce PDF, so i am not quite sure what the actual recommended toolchain is.

I'll track the github issue you opened. Lets see what they say.

Where could i see the SVG conversion ? Curious how well/bad it looks.

Cheers Toerless

On Wed, Mar 30, 2022 at 04:45:39AM -0700, David von Oheimb wrote:

Toerless, thanks for your further response.

I'd just stick to getting the png into the PDF as i originally suggested. No artifical limitations on quality or color, minimal work.

The examples in https://www.ietf.org/archive/id/draft-ietf-pals-congcons-02.xml look like this:

<figure><artwork src="T1E1rvp64.jpg">
<![CDATA[
--------------------------------------------------------------------
I                                                                  I
I             E1/T1 PWs vs. TCP for segment size 64B               I
I                                                                  I
I                     (only in PDF version)                        I
I                                                                  I
--------------------------------------------------------------------
]]>
</artwork>
<postamble> Figure 1  E1/T1 PWs vs. TCP for segment size 64B </postamble>
</figure>

This does give a nice hint how to avoid the trouble and complications I had with <artset> and two <artword> entries. So I improved the XML code to:

<figure title="BRSKI-AE abstract protocol overview">
    <!-- Image converted from PDF using https://www.zamzar.com/convert/pdf-to-svg/-->
    <artwork src="overview.svg">
[ Cannot render SVG graphics - please view
  https://raw.githubusercontent.com/anima-wg/anima-brski-ae/main/overview.png ]
    </artwork>
</figure>

Yet when I try using <artwork> with any binary graphics format (e.g., JPG and PNG), xml2rfc fails with

Error: Discarded unexpected <artwork> content with type='jpg': ''utf-8' codec can't decode byte 0xff in position 0: invalid start byte'
Warning: No image data found in source file overview.jpg

which may due to a bug in current xml2rfc or some Python 3 hick-up.

I worked around this as commented in my new XML snippet, using an SVG file that I meanwhile produce from an intermediate PDF file using https://www.zamzar.com/convert/pdf-to-svg/ This converter copes with a larger number of colors and produces good quality in rather short time.

Using SVG also has the nice merit that HTML output of xml2rfc includes the graphics.

Now, if you're interested to complain about the state of affairs with SVG, there are mailing lists for that as well (also likely to no much effect).

I've just reported an issue at https://github.com/ietf-tools/xml2rfc/issues/751.

-- Reply to this email directly or view it on GitHub: https://github.com/anima-wg/anima-brski-ae/issues/22#issuecomment-1083037982 You are receiving this because you were mentioned.

Message ID: @.***>

--

@.***

DDvO commented 2 years ago

Nice! When i run my local xml2rfc, i can not actually see an option to produce PDF, so i am not quite sure what the actual recommended toolchain is.

You can call it like this:

xml2rfc --pdf -o draft-ietf-anima-brski-ae.pdf draft-ietf-anima-brski-ae.xml

I'll track the github issue you opened. Lets see what they say. Where could i see the SVG conversion ? Curious how well/bad it looks.

Unfortunately, seems that one cannot attach files to GitHub discussions like ours here, but I just put the resulting document at https://github.com/anima-wg/anima-brski-ae/blob/main/draft-ietf-anima-brski-ae.pdf

Cheers, David

toerless commented 2 years ago

Hmm. interesting. But that PDF has full color, so the SVG you generated was probably not IETF SVG compliant ;-)

But looks great. I see no quality deterioration over the original png... ?

Cheers Toerless

DDvO commented 2 years ago

But that PDF has full color, so the SVG you generated was probably not IETF SVG compliant ;-)

Sigh, I did not think of that (IMHO needless) color restriction imposed by RFC 7996. Updated the graphics (and output files) to use b&w only, which makes it less appealing.

BTW, the example RFC you mentioned: https://www.rfc-editor.org/rfc/rfc7893.pdf still makes use of colored graphics - maybe because it sneaked in before 7996 ;-)

But looks great. I see no quality deterioration over the original png... ?

Yeah, the conversion by https://www.zamzar.com/convert/pdf-to-svg/ is very good. One just needs to make sure that its PDF input has sufficient resolution (300 DPI).

DDvO commented 2 years ago

Argh, so far I was able to ignore the xml2rfc error

draft-ietf-anima-brski-ae.xml(4): Error: Did not expect element image there, at /rfc/middle/section[4]/section[2]/figure[1]/artset/artwork[1]/*/*[1]/*

but when uploading the XML file via https://datatracker.ietf.org/submit/ I get the error

/tmp/draft-ietf-anima-brski-ae-ozyli7bb.xml(619): Error: Expected an <artwork> src= file at '/tmp/overview-bw.svg', but no such file exists

When I make the XML file self-contained by directly including the SVG file contents like this (actual image data abbreviated here):

<t><figure title="BRSKI-AE Abstract Protocol Overview">
    <artset>
        <artwork type="svg">
<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" width="946.56pt" height="410.4pt" viewBox="0 0 946.56 410.4" version="1.1">
<defs>
<image id="image5" width="3944" height="1710" xlink:href="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAD2gAAAauCAIAAACl5x6jAAAABmJLR0QA..."/>
</defs>
<g id="surface1">
<path d="M 0 0 L 946.558594 0 L 946.558594 410.398438 L 0 410.398438 Z M 0 0 "/>
<use xlink:href="#image5" transform="matrix(0.24,0,0,0.24,0,0)"/>
</g>
</svg>
        </artwork>
        <artwork type="ascii-art">
[ Cannot render SVG graphics - please view
  https://raw.githubusercontent.com/anima-wg/anima-brski-ae/main/overview.png ]
        </artwork>
    </artset>
</figure></t>

it is rejected with the above xml2rfc error

/tmp/draft-ietf-anima-brski-ae-vwjm_659.xml(623): Error: Did not expect element image there, at /rfc/middle/section[4]/section[2]/figure[1]/artset/artwork[1]/*/*[1]/*
DDvO commented 2 years ago

I experimented with various tools converting images to SVG format. Most of them transform raster images not really to vector graphics, but to pseudo SVG files that still include raster image data, except for https://vectormagic.com/ and https://products.aspose.app/svg/conversion/png-to-svg.

I did not find any tool that produces SVG files (with good quality and) that are accepted by xml2rfc. Even after manually removing inessential elements like <metadata/>, xml2rfc still does not accept style attributes, and these are essential for the given picture, so cannot be stripped.

Giving up on including graphical images.

mcr commented 2 years ago

The SVG image involved stored at o.png is really quite complex, and even if could accomodate full colour SVGs, the diagram would still be way too complex, I think. We probably need a series of diagrams here.... I will buy a bottle of and we can perhaps work together during IETF115 (or hackthon) to beat this into submission. If we won't have quorum, then we should do this over a few BRSKI design team calls. (My apologies for letting this sit for so many months)

DDvO commented 11 months ago

Closing this issue as we likely cannot do anything more about it and do not really need this.