non-Hypertext inside Hypertext, validate discussion

[pzinn]

This is not so much an issue as a call for discussion about the following. Ever since #1288 (2.5 years ago), the role of html has changed quite a bit. It used to be that html would only accept instances of Hypertext as argument, and would produce the corresponding HTML code. After a series of PRs following #1288, html now accepts any type and converts it to HTML code according to the following recipe: by default (in particular for all mathematical types), it does something like htmlLiteral @@ tex where htmlLiteral makes sure characters that have a special meaning in HTML are encoded (the implicit assumption is that something like KaTeX will be running and concert it); for some other types (including, but not limited to, Hypertext) it just produces HTML code directly. There's one problem with this new paradigm which I uncovered as part of my current improvements to locate, code, listSymbols (in relation to #2655 and #2375, among other things): it can lead to Hypertext which is considered illegal by validate. Let me give an example (which is relevant to my new, improved, listSymbols): suppose you want to create an HTML TABLE whose contents can have some mathematical stuff, say a ring QQ[x]. Currently the only way to produce this is with something roughly like

TABLE {{QQ[x]}}

(note that you can't write TABLE {{ html(QQ[x]) }} -- that would doubly htmlLiteral the stuff inside html(...) which in general will not produce the desired result). Now this will work perfectly fine -- except if you tried to validate this (e.g., to include it in documentation), it will tell you QQ[x] has an invalid type inside a Hypertext element. Which is a valid point -- with the "old" logic that is still enforced by validate, one can't have non Hypertext inside Hypertext (well, except strings and options, obviously). So at some point in the near future we will have to make a choice:

• either embrace the new paradigm and accept non-Hypertext inside Hypertext. This is the easier option, and the one I would probably favor, except it means getting rid or changing drastically how validate works. There's a whole bunch of related nasty problems (what if stuff inside a Hypertext is say local variables? can't toExtternalString it...) which will need to be resolved.
• revert to the old way. In a way it might seem like the cleaner solution: force stuff inside a Hypertext element to be converted on the spot to its proper HTML code. As the example above shows, this is not possible at the moment -- it would require some rewriting. Most likely one would need to split html into two parts -- one, say toHtml (or better, reuse the semi-obsolete hypertext) -- a bit like expression, though not exactly the same -- that converts any type to String and/or Hypertext, and then html itself, which as it used to be, only acts on Hypertext (and on strings, htmlLiteraling them). The problem with this second approach is, there's no hope of conversion from HTML to Net (as is done for say the documentation when called in Standard mode) for non-Hypertext inside Hypertext.

Any thoughts? I'll probably provide a truncated version of my upgrades mentioned above as a PR, but I'm definitely stuck in moving forward because of the issue I've just described.

[mahrud]

I don't think I understand this:

(note that you can't write TABLE {{ html(QQ[x]) }} -- that would doubly htmlLiteral the stuff inside html(...) which in general will not produce the desired result).

What's an example? It seems to me like the simpler solution is for TABLE, SPAN, etc. to convert non-hypertext stuff in its input by passing them through tex or html.

Alternatively, validate can make small changes as it reads the input and return corrected output in cases like this. One example would be running html on non-hypertext objects.

[pzinn]

fine, here's another example:

SPAN {<|a|>}

this will work, but SPAN {html(<|a|>)} will fail.

As to changing validate, this might work, but how would you avoid the problem I mention at the end? How can conversion to net work correctly? (without replacing say <|a|> with its TeX version?

[mahrud]

Ahh I see what you mean. Maybe SPAN { tex <|a|> } (or texMath) is the right way to go then?

As to changing validate, this might work, but how would you avoid the problem I mention at the end? How can conversion to net work correctly? (without replacing say <|a|> with its TeX version?

As far as I know, validate is only called in the context of html validate X; that is, right before being converted to html. After all, there are no validation rules for nets, only html files, so we would never pass the output of validate to net.

[pzinn]

OK sounds good -- then maybe I'm worrying about nothing, maybe just relaxing slightly validate along the lines of what you said would be good enough. not closing yet this thread, will keep experimenting.

[pzinn]

BTW, just to give an idea of what I'm trying to achieve and why it involves a lot of non-Hypertext inside Hypertext, see https://www.unimelb-macaulay2.cloud.edu.au/?user=xxx (this is an experimental build) The difference is particularly visible in WebApp mode (note all the clickable links), but should work equally well in Standard mode.

[DanGrayson]

re: "accept non-Hypertext inside Hypertext"

The name "Hypertext" implies that the object should consist just of hyper-text.

[mahrud]

The name "Hypertext" implies that the object should consist just of hyper-text.

Except TEX, ExampleItem, LATER, LITERAL, MENU, TO, etc., so probably an incorrect implication.