first version of xml, xhtml and epub export tagging support

hernot commented 4 years ago

late happy easter

Managed to add taggs and css files to support xml/xhtml export and epub generation

This is a very first version. But line numbering will still need some indeep attention. currently only tested with number location left. Which works outside floats but already beraks inside () And i do expect even wors breaking for any other remaining placement ()

(*) line numbers are not placed immediately infront or after trags but immediately after float tag and before end float tag.

But for some advice it should be fair and OK enough.

adityam commented 4 years ago

This is very exciting! Thanks. I should have time next week, when I'll go over both your pull requests.

adityam commented 4 years ago

This commit only contains the test files. Did you forget to include some files? Regarding linenumbering: If the output is correct without linenumbering, then in the worst case linenumbering can be done at the CSS/HTML level as well. Not the ideal solution, but ....

adityam commented 4 years ago

This commit only contains test files. There is no file which actually implements the tagging. Perhaps you forgot to add it to your branch. Could you please check it.

Thanks

hernot commented 4 years ago

Ouups missed to add them should be solved for now, squash in final commit

adityam commented 4 years ago

Which version of ConTeXt (context --version) are you using?

With MkIV 2020.01.30, the files do not compile. For example:

context tests/vim/38-SYNBOL-SYNEOL-ln-tagging-export.tex

gives

tex error       > tex error on line 8 in file tests/vim/38-SYNBOL-SYNEOL-ln-tagging-export.tex: ! Undefined control sequence

<recently read> \dosynchronizeexport 

\vimtyping@define ...ingparameter \c!alternative }
                                                  \setvalue {\e!start raw#1}...
l.8 ...hon,tab=4,numbering=yes,numberlocation=right]

I get a similar error with LMTX 2020.04.27.

hernot commented 4 years ago

Hi

[Inline Response to message by Aditya Mahajan, Mi, 2020-04-29 14:11 -0700: Answers and comments below corresponding cited paragraphs.]

Which version of ConTeXt (context --version) are you using?

With MkIV 2020.01.30, the files do not compile. For example:

context tests/vim/38-SYNBOL-SYNEOL-ln-tagging-export.tex

gives

tex error       > tex error on line 8 in file tests/vim/38-SYNBOL-
SYNEOL-ln-tagging-export.tex: ! Undefined control sequence

<recently read> \dosynchronizeexport 

\vimtyping@define ...ingparameter \c!alternative }
                                                  \setvalue {\e!start
raw#1}...
l.8 ...hon,tab=4,numbering=yes,numberlocation=right]

I get a similar error with LMTX 2020.04.27.

I currently use 2018 tex-life als 2020 texlife from jonathonf was not yet ready and 2019 was broken. The standalone i did not try yet as i currently use it in production and already had a texlife setup and did not want to switch.

Maybe this would also solve some of my other problems. I will take care about it in a few weeks when i hope to have finished my lecture notes and thus safely can switch to an up to date and clean setup including the recent release of t-vim.

Xristoph

adityam commented 4 years ago

I currently use 2018 tex-life als 2020 texlife from jonathonf was not yet ready and 2019 was broken. The standalone i did not try yet as i currently use it in production and already had a texlife setup and did not want to switch.

Note that it is possible to install context-standalone in parallel to texlive. In fact, you can have multiple versions of context-stanalone in parallel, so you can keep a frozen version for long running projects but at the same time use the latest version for the newer projects.

Maybe this would also solve some of my other problems. I will take care about it in a few weeks when i hope to have finished my lecture notes and thus safely can switch to an up to date and clean setup including the recent release of t-vim.

No worries. In the meanwhile, I'll see if I can get a better understanding of the export mechanism.

hernot commented 4 years ago

Ok just had some time to test home. The source of error is in \setup_export command definee by t-syntax-groups. On about line 326 the \doifmode{*export} clause defines the method inject_css_file which calls at the end (following the comment) \dosynchronizeexport. The last command is (still) known in 2018 release of Context but has been removed in Current release (2020).

By replacing \dosynchronizeexport by \doifdefined{dosynchronizeexport}{\dosynchronizeexport} The error should be solved.

But the exporter seemst to have some problems with newline characters as it complains about possible paragraph mixup and it does not output any tag to html/xhtml/xml strangely. Not sure if related.

adityam commented 4 years ago

If I change that line, I do not get any elements in the exported XML. Did you make any changes in t-filter or t-vim?

hernot commented 4 years ago

Thats exactly the point where I'm currently struggling with 2020 ConTeXt version (in 2018 version it seemed at least still to work by chance). The accompanying pdf looks as should be. The only thing which i do see with my naive trial and error approach is by turning on export trackers is that it has some problem that the code block would sort before the first paragraph and that seems to be disliked by the exporter. That is why i have sent an email concerning the exporter to the mailing list. But I'm not sure if to ask development questions the normal mailing list is the best place. Especially when it comes to the need to understand how low level things like the exporter work or how the document tree looks like it has to process and choke on.

Do you have an advice which channel to use for asking these kind of questions?

adityam commented 4 years ago

On Mon, 18 May 2020, hernot wrote:

That is why i have sent an email concerning the exporter to the mailing list. But I'm not sure if to ask development questions the normal mailing list is the best place. Especially when it comes to the need to understand how lowlevel things like the exporter work or how the document tree looks like it has to process and choke on.

Do you have an advice which channel to use for asking these kind of questions?

The mailing list is the right channel because all the experts are on that list. But you do have to break down the problem into bite sized chunks and include easily reproducible examples.

Please don't take this as a criticism, but in my opinion your latest question did not receive any response because there was no way to reproduce the error message. You had linked to your pull request, but not stated what needs to be run in order to get the error message. Also, asking someone else to discuss internal code in a module can be difficult. If you can translate that into a smaller, stand-alone question, then there is a higher chance of positive replies.

hernot commented 4 years ago

Exactly, i was too clueless and disoriented to be able to get anywhere close to be able to break down to anything helpfull. I do fear for now I'm just the apprentice having to whatch and learn and not much help otherwise. Cause even with the answers you got i'm still as clue less and lost as before.

hernot commented 4 years ago

By the way where do i find documentatoin how to properly debug my contributions, how do is get to know what of my thoughts about context internal workings are wrong and where do i find documentation about the designprinciples of context any of its modules.

Is the mailing list the only source?

adityam commented 4 years ago

On Mon, 25 May 2020, hernot wrote:

By the way where do i find documentatoin how to properly debug my contributions, how do is get to know what of my thoughts about context internal workings are wrong and where do i find documentation about the designprinciples of context any of its modules.

The high-level interface is documented in the manuals. There was a bug which I reported in the mailing list. Hopefully, that will be fixed in the next release.

Is the mailing list the only source?

Yes. Mailing list + reading the source (which used to very easy to read pre-luatex, but now is more complicated because functionality is split between lua and tex).

The actual code for tagging in exports is pretty simple. I pushed a new branch called export which works with my bugfix. The only remaining issues are linenumbering and maintaining spaces (which already happens for \starttyping, so it is just a question of mimicking that functionality).

I am waiting for a new release with the bugfix, so that I can then use to test further.

Aditya

adityam commented 4 years ago

Okay, first step towards the missing features. I pushed a new version 4772457c9bb8252810653daba83fc707202a5c4a, which now fixes the spaces in export. Now looking at line numbers.

adityam commented 4 years ago

Linenumbering is also sorted. Was a backend issue. I am waiting for the context distribution to be updated and then we can test the export functionality.

adityam commented 4 years ago

See branch export-2. It works here for simple examples (with some local changes to strc-tag.lua and back-exp.lua). For example, the file:

\setupbackend[export=xml]

\usemodule[vim]

\definevimtyping[PYTHON][syntax=python]

\starttext
\startsection[title={A code listing}]
  \startparagraph
    This is a code listing
  \stopparagraph
  \startPYTHON
    # Python program listing
    def foobar
        print("Hello World")
  \stopPYTHON

  \startparagraph
    Now the same example with line numbering
  \stopparagraph
  \startPYTHON[numbering=yes]
    # Python program listing
    def foobar
        print("Hello World")
  \stopPYTHON
\stopsection
\stoptext

gets exported to

<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>

<!--

    input filename   : test
    processing date  : 2020-05-31T14:19:54-04:00
    context version  : 2020.05.25 23:39
    exporter version : 0.35

-->

<!-- This export file is used for filtering runtime only! -->

<document context="2020.05.25 23:39" date="2020-05-31T14:19:54-04:00" file="test" language="en" version="0.35" xmlns:m="http://www.w3.org/1998/Math/MathML">
 <metadata>
 </metadata>
 <section detail="section" chain="section" implicit="1" level="3">
  <sectioncaption>
   <sectionnumber>1</sectionnumber>
   <sectiontitle>A code listing</sectiontitle>
  </sectioncaption>
  <sectioncontent>
   <paragraph>This is a code listing</paragraph>
   <vimtyping detail="PYTHON">
    <verbatimline><syntaxgroup detail="Comment"># Python program listing</syntaxgroup></verbatimline>
    <verbatimline><syntaxgroup detail="Statement">def</syntaxgroup> <syntaxgroup detail="Function">foobar</syntaxgroup></verbatimline>
    <verbatimline>    <syntaxgroup detail="Function">print</syntaxgroup>(<syntaxgroup detail="String">"</syntaxgroup><syntaxgroup detail="String">Hello World</syntaxgroup><syntaxgroup detail="String">"</syntaxgroup>)</verbatimline>
   </vimtyping>
   <paragraph>Now the same example with line numbering</paragraph>
   <vimtyping detail="PYTHON">
    <verbatimline><linenumber>1</linenumber><syntaxgroup detail="Comment"># Python program listing</syntaxgroup></verbatimline>
    <verbatimline><linenumber>2</linenumber><syntaxgroup detail="Statement">def</syntaxgroup> <syntaxgroup detail="Function">foobar</syntaxgroup></verbatimline>
    <verbatimline><linenumber>3</linenumber>    <syntaxgroup detail="Function">print</syntaxgroup>(<syntaxgroup detail="String">"</syntaxgroup><syntaxgroup detail="String">Hello World</syntaxgroup><syntaxgroup detail="String">"</syntaxgroup>)</verbatimline>
   </vimtyping>
  </sectioncontent>
 </section>
</document>

hernot commented 4 years ago

Hi adityam

Ok that i would have never sorted out, given the necessity of contibutitions to core ConTeXt. Too many improvements there, that making vimtyping output appear nicely in XML and E-Pub output would just be some adoption of module code only.

Thank You very much for taking over.

I's there anything left, after new version of ConTeXt including your patches has been released, for me to support you eg. helping in writing Tests etc. ? Otherwise if nothing is left for me to support with, I'd suggest to close my (this) pull-request and continue with your branch.

Best Xristoph

adityam commented 4 years ago

On Mon, 1 Jun 2020, hernot wrote:

Ok that i would have never sorted out, given the necessity of contibutitions to core ConTeXt. Too many improvements there, that making vimtyping output appear nicely in XML and E-Pub output would just be some adoption of module code only.

Thank You very much for taking over.

I's there anything left, after new version of ConTeXt including your patches has been released, for me to support you eg. helping in writing Tests etc. ?

I still haven't looked at generating CSS. Since the module only provides a two built-in color schemes, I was actually thinking of including a hand coded CSS rather than an automatically generated one. But I need to look into that.

I'd also appreciate your help with testing to make sure there is no corner case.

Thanks, Aditya

hernot commented 4 years ago

I still haven't looked at generating CSS. Since the module only provides a two built-in color schemes, I was actually thinking of including a hand coded CSS rather than an automatically generated one. But I need to look into that.

Unless changes in colour management are as radical as for tagging in latest mkiv and lmtx than in theory my suggestion on how to create css from colourscheme setup in context and hook it to css output list should still work. In other words no need for static css and for special cases users still can manually attach css through cssfile option of export.

I'd also appreciate your help with testing to make sure there is no corner case.

Ok please keep me posted on progress and on which dedicated tasks or issues my support would be needed and/or apriciated.

adityam commented 4 years ago

The code now works with the latest version of ConTeXt. Can you test the export-2 branch. See tests/41-export.tex for a simple example.

Please note that I have not yet added the CSS functionality (haven't had the time). I'll try to test your code and then add something appropriate. Note that ConTeXt does generate a bare-bones CSS definitions in *-epxort/style/filename-template.css.

hernot commented 4 years ago

Yes i will test with latest an report any

Yes i know about *-epxort/style/filename-template.css. I decided against it for the following probably selfish reasons.

It is not automatically included in css used by output and has to be added manually through cssfile parameter of export by user.
I have no idea if it is recreated on every run or only created if not present.
If recreated only if missing than i consider it to be a reminder for the user to collect all his specific css stuff which is not provided by ConTeXt or any of its packages therein.
In case recreated every time why user still has to add manually? And what if user does not add, what does that mean for typesetting of code?
What side effects would occur if t-vim would automatically add to ensure for proper display of code?

Given these i could not fully figure i decided against utilizing this file for placing vimtyping related stuff. And yes I admit i might have been a bit lazy and lousy to not approach those via Mailing list who could have helped to obtain more clarity upon above points ;-)

hernot commented 4 years ago

Ok seems to work eventhough all syntaxgroups are for now maked as verbatim lines. Are there any obstacles where your expertise would be needed or is that something i could play with at least?

Just for understanding: shouldn't 41-export.tex be placed in tests/vim as it is a tests for vimtyping and not basic t-filter? or is vimtyping just the vehicle or better the environment allowing to test the underlying changes to t-filter?

adityam commented 4 years ago

Ok seems to work eventhough all syntaxgroups are for now maked as verbatim lines.

What do you mean by that? Here I get:

  <vimtyping detail="PYTHON">
   <verbatimline><syntaxgroup detail="Comment"># Python program listing</syntaxgroup></verbatimline>
   <verbatimline><syntaxgroup detail="Statement">def</syntaxgroup> <syntaxgroup detail="Function">foobar</syntaxgroup></verbatimline>
   <verbatimline>    <syntaxgroup detail="Function">print</syntaxgroup>(<syntaxgroup detail="String">"</syntaxgroup><syntaxgroup detail="String">Hello World</syntaxgroup><syntaxgroup detail="String">"</syntaxgroup>)</verbatimline>
  </vimtyping>
  <paragraph>Now the same example with line numbering</paragraph>
  <vimtyping detail="PYTHON">
   <verbatimline><linenumber>1</linenumber><syntaxgroup detail="Comment"># Python program listing</syntaxgroup></verbatimline>
   <verbatimline><linenumber>2</linenumber><syntaxgroup detail="Statement">def</syntaxgroup> <syntaxgroup detail="Function">foobar</syntaxgroup></verbatimline>
   <verbatimline><linenumber>3</linenumber>    <syntaxgroup detail="Function">print</syntaxgroup>(<syntaxgroup detail="String">"</syntaxgroup><syntaxgroup detail="String">Hello World</syntaxgroup><syntaxgroup detail="String">"</syntaxgroup>)</verbatimline>
  </vimtyping>

Note that each line is marked as a <verbatimline> and each syntax group is marked separately inside it.

Are there any obstacles where your expertise would be needed or is that something i could play with at least?

As far as the XML is concerned, I think that this is okay. So, please feel free to play around with this.

I haven't had the chance to look at the CSS part yet.

Just for understanding: shouldn't 41-export.tex be placed in tests/vim as it is a tests for vimtyping and not basic t-filter? or is vimtyping just the vehicle or better the environment allowing to test the underlying changes to t-filter?

That was a mistake. Thanks for catching it. Fixed.

hernot commented 4 years ago

Ah yes you are right should have read carefully. And the rest is a matter of css. In html it appears with every elment on a single line as if firefox would only see block setting for vimtyping and not get the css div entries.

whereas in xhtml it seems to recognize the corresponding css settnings.

As far as i remember i had similiar struggles but i think it should resolve by dropping div, but I'm not quit sure how i got rid of that strange behaviour anymore. I just remember that it was gone somehow.

If you want to compare i think you would have to replace syntaxline by verbatimline. But not sure whether this helps.

adityam commented 4 years ago

I don't have a good understanding of CSS and HTML. I'll let you figure that out :-)

⁣

On Jun 17, 2020, 2:59 PM, at 2:59 PM, hernot notifications@github.com wrote:

Ah yes you are right should have read carefully. And the rest is a matter of css. In html it appears with every elment on a single line as if firefox would only see block setting for vimtyping and not get the css div entries.

<img src="https://user-images.githubusercontent.com/11776325/84936635-d61ff880-b0da-11ea-9cde-f1edf617db89.png" width=350>

whereas in xhtml it seems to recognize the corresponding css settnings. <img src="https://user-images.githubusercontent.com/11776325/84936742-02d41000-b0db-11ea-9bb0-3faffb7d5544.png" width=350>

As far as i remember i had similiar struggles but i think it should resolve by dropping div, but I'm not quit sure how i got rid of that strange behaviour anymore. I just remember that it was gone somehow.

-- You are receiving this because you commented. Reply to this email directly or view it on GitHub: https://github.com/adityam/filter/pull/34#issuecomment-645561003

hernot commented 4 years ago

Ok I'll re figure and pullrequest if necessary against your branch as soon as i think i do have a usable solution.

EDIT The reason why not respecting css is that the templates have to be added manually by including in the preamble the following line

\setupexport[cssfile=41-export-templates.css]

That was one reason why i added the automatic adding of vimtyping styles, as either user has his own custom css not related at all to vimtyping or has no clue how to get colorscheme right.

The following finding or suggestions i do have but for some would need your further help to make work properly at least for some.

1) i would use the colorscheme as detail to vimtyping tags as language is already converted into <syntaxgroup detail=....> </syntaxgroup> or <div class="syntaxgroup ..."> </div> whereas color scheme would be lost. And in css it is possible to switch styles for syntaxgroup dependent upon within which outer tag it is enclosed.

2) xhtml and html behave differntly when you set outer container (eg. <vimtyping detail="Python"> </vimtyping> ) display to display=block. and <verbatimline> </verbatimline> to display=inline

In html it is necessary to end each syntaxline by following it by a <div class="break"></break> defined by context core to tell html to break line not sure if </br> also would work.

In xhtml this is not necessary.

I think here you will have to look into, how to tell core to inject either </br> tag or the <div class=break> tags when outputting html but not when outputting xhtml.

3) xhtml refuses to keep line numbering which is written directly in context of <vimtyping detail=Python> </vimtyping> tag on the same line as the <verbatimline> </verbatimline>

Possibly line number injection should be changed a bit that linenumbers are placed within dedicated <div> tags. Not sure what the reason is why this is not done. Maybe Hans and others had some reasons for. I think this is also an issue for you.

4) Attached the css modified such that it looks reasonable on xhtml. 41-export-templates.css.gz As you can see i removed the div and defined the styles for classes only. And i limitted the verbatimline to specific vimtyping. and the same can be done for all the syntaxgroups to allow several colorschemes in the same document. But as soon as the other stuff is identified and possibly a solution could be found i can take care of proper css injection and creation/default styling.

EDIT I think i managed for html with css only without explicit break tags see second version of css (at least in firefox) 41-export-templates.css.gz

But for xhtml i do fear that it would be necessary to enclose line numbers in a <linenumber> tag, possibly enclosing both number and <verbatimline> tag, or include them within the <verbatimline> tag to ensure that both stay on the same line. But for this i need some time as I'm worse in xhtml + css than html+css.

EDIT Why so complicated. To get proper line breaks just set white-space property of vimtyping to white-space: pre; and all lines are typed as ar. Further concerning the XHTML issue. My mistake just typo in the css. Had detail="Python" on the lines limiting formating for verbatimline to when occuring inside <vimtyping> tag instead of detail="PYTHON" as should have been. With these changes both html and XHTML are formated properly:

41-export-templates.css.gz

In other words if you are ok i can take care of proper css. And creating more export tests. For css i would suggest if you are ok the following.:

styles are linked to colorscheme and not language as 2context.vim script anyway creates language independent TeX/ConTeXt code and thus only proper setting of colours fonts and others is required for proper syntaxhighlighting with is per se language independent.
append vimtyping.css files automatically to list of aditional cssfiles specified through cssfile parameter of \setupexport command. Reason: the *templates.css file is not automatically linked into the exported .html and .xhtml files. It seems as it is just what it name says a template ConTeXt'ers could use as a template for their own .css files they can attach through the cssfile parameter which can be a single file or a list of files. And as the colorscheme definition macros allow users of vimtyping to define their own colorscheme or modify some elements thereof. As a consecquence when they will do so they will also expect that their fancy new scheme is used on html, xhtml and epub which is based upon xhtml. And event if i select one of the two predefined i would expect that this is used on html and xhtml export too without having to manually link, write or even know any css at all as i do not have to on pdf either. Just my approach and opinion.
create a separate .lua file for all things to be handled in lua and link it to vimtyping related files instead of inlining luacode within tex and mkiv files.
do you know is there anything special to be done for lmtx or is lmtx automatically falling back to mkiv if no lmtx specific files and commands are found? Or should that be postponed still for future?

hernot commented 4 years ago

I'm nearly there to make things work as expected for proper display of typeset syntax in export to html, xhtml and xml. Only one little thing is to be solved where i do need your advice and guidance how to make it work without breaking anything.

The t-vim module is based upon t-filter and the outer most set of <div> tags for each syntax display is placed by t-filter. It assigns to the outer most <div> tag on line 488 in t-filter.tex

\dostarttagged{\externalfilterparameter\c!taglabel}\currentexternalfilter

The class name is set to the passed taglabel parameter and tag detail specifier is set to the content of \currentexternalfilter. For vimtyping this holds the name of the language typeset eg. "PYTHON". But for css styling the name of language has no meaning at this stage. The language has already been converted into language independent structure of <div class="verbatimline"> and <div class="syntaxgroup [syntaxelement]"> tags throug 2context.vim script.

Therefore the tag for vimtiyping should better use colorsheme name as tagdetail information like so

<div class="vimtyping pscolor">

and in general

<div class="vimtyping [colorscheme]">

How should i, may i or do i have to modify or extend the related line in t-filter.tex to be able to select an alternative value for the tag detail value instead of the value stored in \currentexternalfilter. Thereby i want to avoid no non desired side effects on other external filters defined elsewhere occur.

adityam commented 4 years ago

I can think of two possibilities:

I add a parameter, tagdetail and use

\dostarttagged{\externalfilterparameter\c!taglabel}{\externalfilterparameter\c!tagdetail}

Then t-vim can set tagdetail to the name of the colorscheme (or perhaps both colorscheme and language; I have seen CSS with class="name1 name2 name3").
We use \startelement[taglabel][colorscheme=name-of-colorscheme, language=name-of-language]. This is nicer because we get a more detailed XML, but it will require some refactoring as the current interface doesn't allow passing the list of key-value pairs). One also needs to globally enable \setupelements[properties=yes] to get the tags, and I am not sure how this interacts with other modules.

So, I am leaning towards the first, as it is trivial to implement.

hernot commented 4 years ago

which ever option you prefer. Eventhough how often one really utilizes different colourscemes for listings of the same language. I think lets start with option 1 and if usecases occur for second than refactoring can be done. Or do you already have existing documents where colorscheme and language should be included within tagdetail?

Concerning \setupelements[properties=yes] is that specific to t-filter and t-vim or a global for ConTeXt. In case the latter i can check if that is set automatically when \setupbackend and \setupexport commands are used.

adityam commented 4 years ago

I'll implement something with the first option.

\setupelements[properties=yes] is global for everything. The way the context backend code is written, it is not possible to localize it effect.

hernot commented 4 years ago

Are you sure about \setupelements[properties=yes] i did not find it neither in docs nor in the standalone code and docs? Where is it mentioned?

adityam commented 4 years ago

Sorry,

\setupexport[properties=yes]

or

\setupexport[properties=prefix]

hernot commented 4 years ago

Ah now i get what you mean, but from reading epub-mkiv.pdf I'm not sure if that is relevant for anything beyond structures created through\startelement \stopelement at least that is how i would read last section 6. As it is only mentioned there and all examples shown there use \startelement \stopelement. And first Paragraph in this section says

"[...]The default output reflects the structure present in the document. If that is not enough you can add your own structure [...]"

And as i view it t-filter and t-vim output is structure which is already present in the document. So I'm not sure if the properties parameter of \setupexport would have any effect to its tags and how it would be possible to make it have an effect. And would that be necessary at all?

Cause epub is basically zip archive including all files required to properly display the document, with the xml describing the pages, and some more informatoin hosted by additional meta files. And xml and xhtml do not differ that much.

Maybe Hans can shed some light on it.

adityam commented 4 years ago

There are two ways to implement tagging.

The first, which we currently follow in t-filter/t-vim is

\dostarttagging{name}{value}
...
\stopstoptagging

which gets translated to

<name detail=value>
...
</name>

The other option is

\startelement[name][option1=value1, option2=value2]
...
\stopelement

By default, this gets translated to

<name>
...
</name>

But, if \setupexport[properties=yes] is set, this gets translated to

...

However, if \setupexport[properties=prefix] is set, this gets translated to

...

I get the impression that \dostarttagging is for internal code while \startelement is for user code. The limitation of \dostarttagging is that we can only have one option. (There is also something which captures the inheritance structure with chain, but that might not be relevant for us).

If we use \startelement in t-filter, then (i) we need to enable \setupexport[properties=yes] but this might interfere with user code. For example, in a personal document, I might or might not want properties to be enabled globally. (ii) If the user sets \setupexport[properties=prefix], then the output of t-filter/t-vim gets affected. I don't like that.

I have implemented a version in export-2 using \dostarttagged so that the output is

<vimtyping detail="name-of-colorscheme>
....
</vimtyping>

hernot commented 4 years ago

So we do have the same impression that \startelement \stopelement is for document editors which want to add custom elements which have no corresponding item or element in other output formats like PDF. I would opt for sticking with \dostarttagged \dostoptagged for now as it is more consistent with standard ConTeXt and has less side effects. Could it be that the chain attributed are filled by the third variant \dostarttaggedchained Where the detail specifier is followed by a reference to another tag? Not sure if that has any value for vimtyping. But doesn't that for now deviate a bit from the initial idea/request to get comparable results from vimtyping on default pdf output and html, xhtml and xml output. In other words unless you have already some use-case or know somebody who already has, I'm fine with what is implemented so far using \dostarttaged and also fine if for now there would be an optional parameter for providing alternative content of tag detail attribute which as a default contains content of \currentexternalfilter as is now. Less efforts, hassle with sideeffects for you and no dependency upon the good will of user to enable properties on \setupexport and anything else would have to be coordinated and thoroughly discussed and designed together with Hans and others as this would mean an extension to ConTeXt exporter backend and tag system.

adityam commented 4 years ago

So we do have the same impression that \startelement \stopelement is for document editors which want to add custom elements which have no corresponding item or element in other output formats like PDF.

Yes

I would opt for sticking with \starttagged \stoptagged for now as it is more consistent with standard ConTeXt and has less side effects. Could it be that the chain attributed are filled by the third variant \starttaggedchained Where the detail specifier is followed by a reference to another tag? Not sure if that has any value for vimtyping.

I agree.

But doesn't that for now deviate a bit from the initial idea/request to get comparable results from vimtyping on default pdf output and html, xhtml and xml output.

Please look at the current export-2 branch and see if something is missing (apart from CSS). In my opinion the generated XML is fine. If you agree, then I can merge this to the main branch and then we can see how to include CSS.

hernot commented 4 years ago

Oh i have an uptodate copy of your export-2 on my repo and in my local directory already css included. Just need to run some further tests and than i would place a new pull-request against your export-2 and close this one, if that is ok for you. That pullrequest would than include most if not everything needed for merging into main.

adityam commented 4 years ago

Please place it against export-2. export-2 and dev have deviated, so I'll rebase locally before merging.

hernot commented 4 years ago

OK done by #38.

adityam / filter

first version of xml, xhtml and epub export tagging support #34