Thin XSL layers and a new workflow

[Alex-Jordan]

For a while now I've wrestled with how to get our own needs for this document met, when those needs clash with Rob and David and others. At first our plan was to have an omd branch of MBX. If we have a new feature, or a different way to do something, we run it by the discussion board and try to get it into MBX. Failing that, we would just put it into our omd MBX branch.

The problem is when there is something we clearly want, but the MBX crew wants it in a different way. We spend time making it work for MBX, and we can be happy that we contributed to MBX. But in the end it maybe doesn't work the way we still want it to for precalc.

Lately I've been thinking that we don't need to branch from MBX at all. Rather, we throw in our additional features/modifications to existing features in a thin XSL layer for precalc that sits above MBX. I've accomplished outcomes and try-it-yourself this way already. I think this approach will speed us up. Our features can reside there and do what we need them to do, and when we're all happy with a feature (MBX crew too) we can move it into MBX.

As I mentioned in an email, in a few days I will have parts of Chapter 1 working without mainfile.tex, and still keeping the features we had in our original pdfs. The work that you put into mainfile.tex is still being used, in that I'm able to put parts of mainfile.tex into the thin XSL layer, and other parts into the latex.preamble strings. When I'm ready with this, I'll post a little bit here, but keep full and clear explanations in the readme file. Everything I am talking about is taking place in side branches of precalc and mathbook, so you'll be able to play around with this proposed workflow without harm.

[cmhughes]

This sounds perfect, I'm looking forward to seeing your work on the first parts of Chapter 1.

On Mon, Mar 16, 2015 at 9:52 PM, Alex Jordan notifications@github.com wrote:

For a while now I've wrestled with how to get our own needs for this document met, when those needs clash with Rob and David and others. At first our plan was to have an omd branch of MBX. If we have a new feature, or a different way to do something, we run it by the discussion board and try to get it into MBX. Failing that, we would just put it into our omd MBX branch.

The problem is when there is something we clearly want, but the MBX crew wants it in a different way. We spend time making it work for MBX, and we can be happy that we contributed to MBX. But in the end it maybe doesn't work the way we still want it to for precalc.

Lately I've been thinking that we don't need to branch from MBX at all. Rather, we throw in our additional features/modifications to existing features in a thin XSL layer for precalc that sits above MBX. I've accomplished outcomes and try-it-yourself this way already. I think this approach will speed us up. Our features can reside there and do what we need them to do, and when we're all happy with a feature (MBX crew too) we can move it into MBX.

As I mentioned in an email, in a few days I will have parts of Chapter 1 working without mainfile.tex, and still keeping the features we had in our original pdfs. The work that you put into mainfile.tex is still being used, in that I'm able to put parts of mainfile.tex into the thin XSL layer, and other parts into the latex.preamble strings. When I'm ready with this, I'll post a little bit here, but keep full and clear explanations in the readme file. Everything I am talking about is taking place in side branches of precalc and mathbook, so you'll be able to play around with this proposed workflow without harm.

— Reply to this email directly or view it on GitHub https://github.com/openmathdocs/precalc/issues/15.

[Alex-Jordan]

OK, it's ready for you to try out.

The first thing to say is that you won't like some of the styling. Just rest assured that styling can be changed. For instance, I have frames around the examples right now, more because it helps me during editing. The frames break at awkward places sometimes, so in the end they can go away or me altered somehow.

The second thing is that as you may have seen in the user group, I am separately working on exercises and solutions. So right now there is no hyperlinking of those, but there will be.

1. Go to your MBX repo. If you have a local copy of the branch called feature/latex.style.extra, or remote/omd/feature/latex.style.extra, delete it, because I force pushed some changes after squashing some commits. I won't do that anymore on this branch now that you will be using it.
2. fetch from omd/mathbook, and create a branch to track omd/feature/latex.style.extra, and checkout that branch.
3. Go to your precalc repo. If you have a local copy of the branch called MBX-only, or remote/omdMBX-only, delete it, because I force pushed some changes after squashing some commits. I won't do that anymore on this branch now that you will be using it.
4. fetch from omd/precalc, and create a branch to track omd/feature/latex.style.extra, and checkout that branch.
5. Now either read the readme.md that is in the MBX-only branch, or if you want to dive in to see some output:

xsltproc -xinclude ../xsl/precalc-latex.xsl precalc.xml >> draft.tex 
pdflatex draft.tex 
pdflatex draft.tex 
open draft.pdf

Along with the two issues I mentioned at first, here are some more issues:

• I'd like to build our own XSL for tables, rather than go in the direction Rob is taking that. We don't have to worry about accommodating vertical rules and some other things, so we will have freedom to "do tables right". You can see how we would accomplish this by looking in precalc-latex.xsl, and see how you can write XSL that trumps MBX.
• sidebyside is repeating children of a p at the end of the p. I know you are fixing that; just listing it here.
• try-it-yourself, outcomes, and standout (my replacement for pccspecialcomment) all do not have HMTL counterparts yet. Since these are all presentation environments, and I don't want to dive into CSS now, that's why they are not addressed yet.
• The fact that sidebyside floats is causing problems with using it the way I'd like to. You can see in Example 1.2.1, where it should start "Example 1.2.1 (Unemployment Rate Table).", then give the sidebyside, then pick up with "How is this table...". But instead the sidebyside comes first, then "Example 1.2.1 (Unemployment Rate Table). How is this table..."
• I think we need bigger font in the graphs, controlled in bookinfo.xml.

[Alex-Jordan]

Also, I know you are concerned about a counterpart to \includeonly. It's on my mind too. Right now there are these options:

• focus on HTML instead of PDF for editing, and trust that if referencing works in HTML that it works in PDF too. (If we don't have that trust, then we'd have that issue in the opposite direction.)
• add a feature to MBX so that it would separate .tex output into individual chapter files and use \include. This would be great, but a potential distraction.
• Make a post-processing script that would cut draft.tex up into pieces using \include, and put it into an arara implementation.

[cmhughes]

Thanks for the details; I've been able to follow your instructions and like what I see.

In particular, I do like the xml mark-up that you've implemented; I think that it should scale well and will adapt nicely to other output formats (ePUB, DAISY, etc). The only exception is your use of sidebyside to put a table next to a paragraph, especially given that you are now happy for us to build our own table routine, which can (for example) accept @style=margin (which I did implement previously) but let's not focus on that at the moment.

I most certainly agree that MBX is fantastic for generating chapter content. I can see why Rob would like the option to generate preamble for LaTeX files, but I don't think it is appropriate for OMD. For example:

I am separately working on exercises and solutions. So right now there is no hyperlinking of those, but there will be.

The mainfile.tex already implements hyperlinked problems and solutions, so I'm not sure why this needs to be worked on--why do we need to re-invent the wheel?

Again, I would very much like to generate chapter content using MBX, but the mainfile.tex should stand alone. It is akin to the CSS creation, which is done separately, and is not generated by MBX on every single run. In the context of the content you've written, function-basics.xml would create function-basics.tex which only contains chapter content.

Perhaps I've misunderstood your intent, if so I apologise--I'd very much like to hear more about why mainfile.tex needs to go, and if it does need to go, I'd like to know why we don't generate CSS files on every translation. In my mind, mainfile.tex is to the LaTeX style as CSS is to the html style.

[Alex-Jordan]

I have some responses that I'll email, because it's just too long I think for this forum.

But in the end I believe we actually go both ways on this issue. If you examine the chunking scheme for HTML (which produces multiple interlinked .html files) and implement it for .tex too, then running xsltprocwould produce:

precalc.tex <includes the chapter.tex files>
chapter1.tex
chapter2.tex
...

I think Rob indicated he would support this if it was just an option controlled by a stringparam switch.

Then I could execute pdflatex precalc.tex, and you could execute pdflatex mainfile.tex. Your pdf output will look better than mine for sure. But I suspect I'll be able to add new features more easily, like the MBX-WBWK bridge I have in mind. As long as source goes into section.xml files like what you see in the MBX-only branch, the two approaches wouldn't have incompatibility issues.

[cmhughes]

Hi Alex, I've been working on the latex side of things on the MBX-only branch ( https://github.com/openmathdocs/precalc/tree/MBX-only); if you pull from it, you should see two new commits from me:

• the first commit allows the user to specify their path information in an ignored/untracked file, xsl/precalc-paths.xsl. You will need to create this file in your directory to point to your mathbook-latex.xsl file. This should allow easier portability.
• the second commit introduces src/mainfile.xml which is quite similar to your original file, src/precalc.xml. Note that BOTH EXIST AT THE SAME TIME!!! The only difference is that src/mainfile.xml doesn't have all of the stuff that MBX uses for frontmatter, backmatter, preamble etc. You'll notice that I've added a small number of modified templates to xsl/precalc-latex.xsl which are only called when @style='chunk' is used, as it is in mainfile.xml.

I believe that this allows us to continue to develop our chapter XML files collaboratively, while allowing two different LaTeX versions. If/when we get to a near-finishing point of this project, we can worry about which LaTeX version to release.

I've done a fair amount of testing, but I might need to add further small templates to xsl/precalc-latex.xsl as and when the need arises; some of it is difficult to predict at this stage.

Does this sound reasonable? If so, perhaps we can start to gather momentum once more; as I said above, we can continue to push forward with chapter files regardless of everything else. I'm also confident that I can incorporate any feature you want into mainfile.tex.

Chris

On Fri, Mar 20, 2015 at 6:20 PM, Alex Jordan notifications@github.com wrote:

I have some responses that I'll email, because it's just too long I think for this forum.

But in the end I believe we actually go both ways on this issue. If you examine the chunking scheme for HTML (which produces multiple interlinked .html files) and implement it for .tex too, then running xsltprocwould produce:

precalc.tex chapter1.tex chapter2.tex ...

I think Rob indicated he would support this if it was just an option controlled by a stringparam switch.

Then I could execute pdflatex precalc.tex, and you could execute pdflatex mainfile.tex. Your pdf output will look better than mine for sure. But I suspect I'll be able to add new features more easily, like the MBX-WBWK bridge I have in mind. As long as source goes into section.xml files like what you see in the MBX-only branch, the two approaches wouldn't have incompatibility issues.

— Reply to this email directly or view it on GitHub https://github.com/openmathdocs/precalc/issues/15#issuecomment-84093218.

[Alex-Jordan]

This sounds great. I'll pull and try it out over the weekend. Thanks!

By the way, in an email recently Rob said he is still thinking about chunking chapters in the LaTeX side. So that may happen too someday.

I've been busy with the calc lab manual. It's coming along.

[cmhughes]

Sounds good; I probably won't work on it much until I hear back from you.

If you are happy with it, this might be the time to merge it into our develop branch so that we can get going with content generation again.

To test it, you run the same command mainfile.xml that you would on precalc.xml; it should produce chapter files in .tex format, no preamble in site!

On Thursday, April 9, 2015, Alex Jordan notifications@github.com wrote:

This sounds great. I'll pull and try it out over the weekend. Thanks!

By the way, in an email recently Rob said he is still thinking about chunking chapters in the LaTeX side. So that may happen too someday.

I've been busy with the calc lab manual. It's coming along.

— Reply to this email directly or view it on GitHub https://github.com/openmathdocs/precalc/issues/15#issuecomment-91338191.

[Alex-Jordan]

I renamed precalc-paths.xsl to precalc-latex-paths.xsl, and created precalc-html-paths.xsl in parallel. In precalc-html.xsl, it loads mathbook-html.xsl, so a similar local path to mathbook-html.xsl is needed.

Everything works for me just as it did with precalc.xml, with precalc on the MBX-only branch, and mathbook on the features/latex.extra.style branch. Also, it works as I would expect it to with mathbook on the dev branch.

I tried using mainfile.xml as:

xsltproc -xinclude ../xsl/precalc-latex.xsl mainfile.xml >> draft.tex

But it produces a blank draft.tex file. What am I doing incorrectly?

Otherwise, seems pretty solid. This week I'll pay attention to the functions chapter and see if I can convert at least one more section of it.

Would you now want to move mainfile.tex back into the src folder? I put it in xsl as something to reference while adding more xsl to precalc-latex.xsl, but perhaps it should be moved now.

[cmhughes]

Ok, that all sounds good.

Everything works for me just as it did with precalc.xml, with precalc on the MBX-only branch, and mathbook on the features/latex.extra.style branch. Also, it works as I would expect it to with mathbook on the dev branch.

I had forgotten about the features/latex.extra.style branch; I don't need it for my chunking though :)

I tried using mainfile.xml as:

xsltproc -xinclude ../xsl/precalc-latex.xsl mainfile.xml >> draft.tex

Apologies, I should have said that you simply run:

xsltproc -xinclude ../xsl/precalc-latex.xsl mainfile.tex

This should produce chapter files, according to the xml:id specified in the chapter or, if it doesn't exist, the number of the chapter, e.g chapter-1.tex.

Would you now want to move mainfile.tex back into the src folder? I put it

in xsl as something to reference while adding more xsl to precalc-latex.xsl, but perhaps it should be moved now.

I have moved mainfile.tex into the src folder and pushed, so you should see that when you pull.

If you're happy with this, do you want to merge it into our dev branch?

[cmhughes]

This new workflow seems to work well; we have a thin xsl layer for both LaTeX and HTML, but we still get the benefits of updates from origin/dev (https://github.com/rbeezer/mathbook). So far we have contributed the quantity and sidebyside features. The decimalAlign feature is currently live in OMD, but has not been submitted to origin/dev yet (see https://groups.google.com/forum/#!topic/mathbook-xml-support/yCczBgkqIq4).