Support for asciiDoc in the description attribute of the info and pathItem Object

[miqui]

Support for asciiDoc in the description attribute of the info and pathItem Object

[MikeRalphson]

Can you give any advantages of making such a change? Is asciidoc better standardised than CommonMark for example or allow things CommonMark does not?

How would we indicate whether the description was to be interpreted as asciidoc or CommonMark?

[briandominick]

I would also like to see AsciiDoc support in OAS converters, since AsciiDoc provides no obvious advantage in terms of markup notation (other than using one * instead of two for bold, and I slightly prefer the link notation -- these seem really small).

The problem is, for a team that uses AsciiDoc for the rest of its documentation, having one case where CommonMark is the required standard is frustrating and just untidy. It isn't really whether one of them does the job sourcing rich-text output in REST API docs; it's just a mess to mix standards.

I think the solution is for tooling to add support, which the spec clearly allows:

Where OpenAPI tooling renders rich text it MUST support, at a minimum, markdown syntax as described by CommonMark 0.27.

[Emphasis added.]

So any tooling could provide support for AsciiDoc and allow a global setting to override the CommonMark default during a conversion. It would be good if the spec facilitated indication of the markup format, at least globally if not per-field (which seems unnecessary). But this doesn't seem strictly necessary for a reasonable solution to the problem.

[ggrossetie]

It would be good if the spec facilitated indication of the markup format, at least globally if not per-field (which seems unnecessary).

👍

But this doesn't seem strictly necessary for a reasonable solution to the problem.

My understanding is that it's necessary to add an indication of the markup format used since the tooling cannot automatically infer it.

[MikeRalphson]

Rather than push the burden of support onto all tool vendors, if your use case is that you use asciidoc elsewhere then I think a pre-processing / conversion step before you publish your OAS documents would be the way to go. asciidoc to docbook and docbook to markdown seems a common approach.

[briandominick]

I don't think I understand this @MikeRalphson. The whole purpose of a spec is to define the "burden" that will be "pushed" onto tool maintainers, if you want to phrase it that way. Right now the spec mandates that compliant projects support CommonMark. That means every vendor MUST include a CM converter -- a "burden". Meanwhile, I am by no means suggesting a support requirement for AsciiDoc. I am actually trying to avoid that burden altogether by interpreting it as optional even without a flag in the schema.

In fact, the spec requirements for description fields themselves (not just the tooling) right now all say that users MAY use CommonMark markup, but without a way to indicate otherwise, users are always using CommonMark by default. So you'll have to use escapes to express literally * something in asterisks*. So the spec already has an apparent weakness if it does not provide a means to select plaintext vs CommonMark but tells users they MAY use CM; effectively right now one MUST write in CM, because the converter is going to parse as if one is writing in CM, without a way to flag otherwise.

To be clear, my conclusion is that tooling providers MAY support AsciiDoc conversion now, according to the spec as it is written. Mike, you seem to be suggesting that users bear the burden instead of vendors, which I don't really understand. But this honestly seems like any new feature that some vendors might support and others would not be required to, all within the spirit of the spec, in the interest of solving real problems for real users. This solution seems more servicable than suggesting every user must hack together a preprocessor for the description fields from AsciiDoc to Markdown inside a data object, which is IMO an even less optimal way to solve this.

[briandominick]

My understanding is that it's necessary to add an indication of the markup format used since the tooling cannot automatically infer it.

@Mogztter I just realized that the OAS3 spec instructs users that they "MAY" use CommonMark, but without a means of indicating plaintext, it really means all description text WILL be parsed as CM. So while I'm sympathetic to this ideally getting a solve in the specified schema, it seems that if plaintext support is not required of tooling, the spec already has a keen contradiction in it around markup. Whatever the spec may seem to be saying, in real life, tooling vendors would be doing everyone a favor if they enabled configuration of if/how to parse description fields.

[MikeRalphson]

@briandominick I'm aware of the requirement for tools to support CommonMark, what I personally do not want, nor see the need for is to add additional requirements which merely provide alternatives, not concrete benefits.

To be clear, my conclusion is that tooling providers MAY support AsciiDoc conversion now, according to the spec as it is written

See https://github.com/OAI/OpenAPI-Specification/issues/1867#issuecomment-473134007

@darrelmiller: That clause was intended to give tooling the freedom to support additional markdown syntax on top of CM 0.27

My take is therefore that alternative rich-text formats such as asciidoc are not supported.

you seem to be suggesting that users bear the burden instead of vendors, which I don't really understand. ... than suggesting every user must hack together a preprocessor for the description fields from AsciiDoc to Markdown inside a data object

If you think I am suggesting that then you have misunderstood. Surely only one tool would ever be required to convert OAS documents using asciidoc to CommonMark so they can be consumed by the rest of the OpenAPI ecosystem? If there is a need for such a tool, it will surface.

[briandominick]

I really cannot understand your conclusion based on that very quote, unless you're interpreting the word "markdown" (lowercase) to suggest they mean "Markdown (uppercase) but not other lightweight markup", which would be beyond silly. I think they meant "markup", not "markdown", in that sentence. Nobody claiming to want "freedom" for tools mean "only if tool maintainers choose one of the 50 flavors of Markdown but no other lightweight markup". Makes no sense.

Surely only one tool would ever be required to convert OAS documents using asciidoc to CommonMark

But since AsciiDoc has a lot more elements than CommonMark, this would be a dreadful non-solution. The markup would lose most semantic notation and whole features like tables, variables, conditionals, etc. This just isn't a solve.

The solution is obviously to let tool providers optionally convert AsciiDoc markup, which is presently how the spec reads. So if a tool provider offers a configuration setting to treat description fields as AsciiDoc sourced, and to provide contextual tooling for AsciiDoc-formatted source like environment variables, that is within the current spec that does not restrict the markup format, in favor of tooling "freedom".

If there is a need for such a tool, it will surface.

There is no market for a tool that takes all that is good and wholesome out of AsciiDoc and renders any form of Markdown, and I'm not holding my breath. But I already have a small market for a tool that supports AsciiDoc inside OAS3 description fields and converts it directly to HTML, so I'll probably just extend my clients' favorite OAS3 docs generator and see if I can't slip an AsciiDoc converter in, as well as a way to indicate the chosen markup format.

[briandominick]

Hmmm, upon rereading, maybe Markdown is really what they meant by "additional markdown syntax". But then my point stands about why would someone proclaim wanting to provide "tooling freedom" but then declare they can only use a format that calls itself Markdown? You either believe in "tooling freedom" or you don't. But anyway, I believe there is a decent resolution for now, and I hope OAS initiative folks consider opening this up more explicitly and letting users indicate markup format, and let toolmakers support formats other than CM, optionally.

[ggrossetie]

I'm aware of the requirement for tools to support CommonMark, what I personally do not want, nor see the need for is to add additional requirements which merely provide alternatives, not concrete benefits.

I think that's quite the opposite. Currently the requirement is that you MUST write in CommonMark. As an example, if you just want to write in plain text, at some point the CommonMark syntax will interfere. For instance, here the * symbol will be rendered as a list but that's not what I want:

Plain text

You should use a* or b**.

* "a" is used to do X.
** "b" is used to do Y.

Result (wrong)

You should use a* or b**.

• "a" is used to do X. ** "b" is used to do Y.

So the idea is not to add an additional requirement but rather to relax the current requirement to give more flexibility/freedom to the tooling. The specification might read as follows:

the OpenAPI tooling MUST be able to render markdown syntax as described by CommonMark 0.27. OpenAPI tooling MAY also support additional markup languages as specified in the OPTIONAL markup field (default value is "commonmark").

With the above definition, an OpenAPI tool CAN support additional markup languages (or just plain text) but that's not a requirement.

[darrelmiller]

@briandominick Are you aware of a tooling creator who sees a customer need and is prepared to add Asciidoc support to their tools?

Without commitment to build tooling support, adding features to the spec is worthless at best and potentially harmful to the ecosystem because it affects customer expectations of existing tooling.

Adding features to the spec and expecting the those building the tools to run out and implement it "because it is in the spec" is something we try and avoid.

[briandominick]

You're misunderstanding me, @darrelmiller. I thought I made it really clear that I was not suggesting a change to the spec other than to clarify the thing highlighted in your other comment. And the very most you can interpret from my suggestion about enabling other languages in description field was that I thought it would be a good OPTION. I was super clear about that, so there's really no reason to straw man my position. I fully agree with you that adding a tooling burden to the spec unnecessarily makes no sense. I would never suggest otherwise.

However, you have a basic tooling burden in the spec, and the spec contains a big contradiction between requiring CM support "at a minimum" and saying users "MAY" write with CM syntax. As @Mogztter aptly illustrated, as it stands users effectively MUST use CM syntax and not plaintext, so having a way to indicate how the description field should be interpreted would help resolve that.

HOWEVER, that was never my main point. I was just saying that yes I have clients who want to convert from Markdown (or anything else) to AsciiDoc but run into an obstacle figuring out if their OAS3 source may or may not use AsciiDoc in description fields. This isn't an issue of the clients wanting to adhere 100% to the spec, but wanting any tool that enables them to use OAS3 with AsciiDoc description fields. And while I'm not suggesting any toolmaker has to support this, or any such silliness you want to attribute to me, consistently I am saying I believe it is within the spirit of the OAS3 spec to allow AsciiDoc in addition to CommonMark. And if you only intend to use AsciiDoc and never CommonMark, then a way to indicate this at conversion time should suffice.

I'm happy to hear your opinion on this since you seem to know what was intended by this clause. I just wanted to be clear that all I was saying was that I actually think your awesome spec comes close enough to handling this for my needs as a tooling consultant. I am now looking at what it would take to extend a FOSS generator to convert AsciiDoc in addition to CommonMark.

[handrews]

@miqui I think we should close this, as our focus in 3.x is on minimal additional tool developer burden (because 3.1 was... a lot). Everyone has their favorite flavor of simpler-than-HTML formatting, and I don't see major tooling vendors as likely to take on an open-ended number of formats. And I don't think having just two would make sense - either there's one, or it's extensible. We can, of course, revisit text formatting in Moonwalk if you want to focus on it there.

[ggrossetie]

And I don't think having just two would make sense - either there's one, or it's extensible. We can, of course, revisit text formatting in Moonwalk if you want to focus on it there.

@handrews I agree that supporting two arbitrary markup languages doesn't make sense. However, I do think that making it extensible is compelling. What is your opinion about my proposal?

So the idea is not to add an additional requirement but rather to relax the current requirement to give more flexibility/freedom to the tooling. The specification might read as follows:

the OpenAPI tooling MUST be able to render markdown syntax as described by CommonMark 0.27. OpenAPI tooling MAY also support additional markup languages as specified in the OPTIONAL markup field (default value is "commonmark").

With the above definition, an OpenAPI tool CAN support additional markup languages (or just plain text) but that's not a requirement. Also, the default value is CommonMark so it's backward compatible.

[handrews]

@ggrossetie it's not about the merits of that idea, it's about what fits in 3.x, and this does not. However, I have filed a discussion for 4.x, where we are able to consider larger-impact changes:

• OAI/sig-moonwalk#128

Our general policy is that if we decide on something in Moonwalk that can reasonably be backported to 3.x, we will backport it, so this does not necessarily preclude a 3.x change.

I'm marking this issue as "moved to moonwalk" and closing it in favor that discussion (@miqui if you disagree please feel free to re-open, I just think given the resistance in some quarters to doing 3.2+ at all, this is too deep in the weeds to be the right cost-benefit tradeoff right now (we can backport if it turns out otherwise).