Add support for Asymptote WebGL

sean-fitzpatrick commented 4 years ago

This adds a new output option for Asymptote in the mbx script. Doing mbx -v -c asy -f html -d foo/ bar will output Asymptote images to WebGL HTML files.

One bug I need to fix before you merge this: If a book has both 2D and 3D Asymptote images, mbx will fail. But I think someone with better Python skills can fix it. The good news is that if asy doesn't detect graph3 being loaded, it reverts to svg as output instead of html. The problem is that mbx script will be looking to copy img_foo.html from the tmp directory, not img_foo.svg, so the script halts with a file not found error.

rbeezer commented 4 years ago

Thanks, Sean. I'll look into fixing up the rough edges on this one. (Classes start tomorrow...)

sean-fitzpatrick commented 4 years ago

OK, great! There is probably no need to change the XSL since we don't really need the html ids. Just need to find a way to tell mbx to look for an svg if it can't find html.

rbeezer commented 4 years ago

In the new Line 112, can you just condition on the existence of the file? In other words, check if

os.path.exists(asyout)

is true prior to attempting the copy? Or am I missing something?

If the outformat is html then that would be all we get? We'd silently ignore the backup/fallback SVG? The author would need to do another run to get those versions. Which, if documented, I think is OK.

sean-fitzpatrick commented 4 years ago

This should be solved with the above commit. The os.path.exists function is what I was missing. Changes:

define asysvg as filebase.svg
if asyout exists, then copy to dest_dir. Else, copy asysvg to dest_dir

MBX will still fail if neither HTML nor SVG exists. But it should, because if neither file exists it means asy failed to compile the source. (asy -f html foo.asy produces foo.html if the line import graph3; is present in the asymptote source, and produces foo.svg otherwise.)

sean-fitzpatrick commented 4 years ago

I should have added: I tested by dropping three each of 2D and 3D asymptote figures into the minimal article and running it through mbx. My images directory promptly gained three 2D svgs and three 3d html files as expected.

rbeezer commented 4 years ago

Excellent. I'll smash the commits together.

Let's coordinate on making some good samples for the sample article, and maybe the Showcase Article. Hitch on my end is that maybe I want to avoid compiling Asy. Or maybe not. ;-)

Does a self-compiled version co-exist with a distribution-package version? Perhaps, say, it installs into something like /usr/local/ or /opt?

I'm going to see if Python has tools to do a version check on the Asy executable. It'd be great if the "html" flag simply failed with an insufficient version!

Thanks for all the work on this - it's going to be a great feature. Certainly for APEX, but also for others.

Rob

sean-fitzpatrick commented 4 years ago

If the asy version isn't recent enough, asy will fail on the html flag. But the error message from asy doesn't make it through to mbx. (usual debugging: run mbx -vv, try compiling the source in the /tmp directory directly, get the error message) Then again, an error message explaining the need to upgrade would be more author friendly :-)

You can download a precompiled binary with the changes John made for us. Compiling from source is a pain -- it took awhile to figure out what libraries and utilities weren't installed.

rbeezer commented 4 years ago

Testing the branch here. Nothing is ever easy. Got bit by #1065. Again.

Once over that, it seems older Asy does not just simply fallback to making SVG. mbx fails and trying the failing command independently gives

/tmp/tmpjD3U8q$ asy -outformat html asymptote-lever-solo.asy
convert-im6.q16: not authorized `asymptote-lever-solo_.eps' @ error/constitute.c/ReadImage/412.
convert-im6.q16: no images defined `html:asymptote-lever-solo.html' @ error/convert.c/ConvertImageCommand/3258.
/usr/share/asymptote/plain_shipout.asy: 87.10: runtime: shipout failed

Investigating version number checking...

rbeezer commented 4 years ago

I think maybe I did not fully understand some combination of source, -outformat, and asy versions.

In any event, I can grab the current version number for Asy and then perhaps print a warning (always). I'd arranging it to occur right before any potential failure due to old versions.

But not tonight. Clocking out.

rbeezer commented 4 years ago

I added a warning, active with use of the -v switch, that reports the version string from asy and says that 2.62 or better is necessary. On its own commit. Compressed commits here, with a bit of whitespace cleanup. Thanks very much, Sean.

svg output format seems to behave as before with my legacy asy (circa 2017). New (previously unimplemented) html output format will fail on a source diagram without graph3. But, the above warning will show up right beforehand.

Pushed, but no updates to anything else. Not closing, there is more to consider.

rbeezer commented 4 years ago

This could be tested by using the mbx switch -r ("root/restrict") to restrict to some division (say) where every Asy diagram is 2D or 3D, but not a mix. At the extreme, root at a "figure", perhaps.

rbeezer commented 4 years ago

Next steps.

We need to protect when the svg format is active and a diagram is 3D and really should be HTML. Perhaps just checking if the right file exists? See #1065
We have the asy source, typically here in asydiagram. Can't we reliably search for the string graph3 perhaps prefaced by a space or a newline? And/or concluded with space, comma, or semi-colon? (I don't know, have not consulted asy source syntax.) Can we be 100% certain a source is meant for 2D or 3D? Then we can be more careful about not trying to make files we know are not going to be made.
I'd like to get some examples in the sample article for testing. I'd try the 2.62 pre-compiled binary.
This seems complicated enough that we need documentation. So time to bite the bullet and start documenting mbx use.

sean-fitzpatrick commented 4 years ago

I forgot that images with colour fill produce an svg outline, and one or more png files to provide the colour. This is fixed with my last commit. Perhaps still missing is a warning message to run with Asymptote 2.62 or newer.

rbeezer commented 4 years ago

I managed to extract ffcc7049106c7a1a23ad371930df8ddd4498dd19 (colour PNGs) and merge that. Very little testing, I still need to get the new binary. Kept the correct spelling of colour in the commit message.

I'll see about getting serious about the import graph3; detection, perhaps by the end of Wednesday.

sean-fitzpatrick commented 4 years ago

"Correct" spelling appreciated! ;-) The only thing I'm wondering about. Say an author (or perhaps publisher) is running mbx -c asy -f svg and there is 3D content.

maybe they're unaware that HTML output is now supported.
maybe they're unaware direct output to SVG is not something asy will do for 3D images.
but maybe they're aware of this, and they just want SVG output, and they're expecting us to have anticipated this and coded mbx accordingly.

So there's an element of mind reading involved in interpreting the choice of svg for output format. In some cases, providing the HTML instead is the right call. In other cases, they really do want the SVG, in which case mbx -c asy -f svg needs to be interpreted as "first do asy -f pdf foo.asy, then do pdf2svg foo.pdf."

I wonder if the default assumption should be that the user really does want svg, and we do the latter (svg via pdf). And then we print a message suggesting that the user consider html as an option.

We could make it work without having the xsl look for use of graph3, but the decision tree gets convoluted (non-python pseudocode coming)

if outformat == 'svg'
  do asy -f svg foo.asy
  if foo.svg exists
    copy foo.svg to dest_dir
  else
    do asy -f pdf foo.asy
    do pdf2svg foo.pdf
    copy foo.svg to dest_dir

or something like that. This would still fail on bad asymptote code and other issues, of course.

rbeezer commented 4 years ago

I've thought about this, too, just recently.

The mbx script should not protect against ignorance. And I like it to run silent without -v or -vv. So perhaps informative messages with -v could be put in place. But this is a job for documentation, which is why I'm prepared to spruce up the general mbx chapter just a bit, and do this carefully, since it is complicated.
That said, I think we should do everything in our power to build what is asked for, without causing predictable errors. In particular, we should aim for not just bailing out when a file is not found - that should happen iff the author's source is busted.
I wrote a template last night to look for import and graph3 as "whole words", both on the same line, using ; as a line separator, not newlines. Works well, except one example in sample article has such an import commented out! I'm not going to worry about that, but will warn in the documentation.
I need to slightly re-architect the extract-asymptote.xsl stylesheet to also create a Python list, and then re-configure mbx to receive it. The will allow the 2D/3D flag to be passed in and we can loop over the list rather than querying the directory for filenames.

So - maybe give me through Wednesday to make above changes (unless you have comments) and then you could reorganize the logic?

Thanks for your work on this - I'm motivated to get this firing on all cylinders, since it will be such a plus for output in books like APEX.

Rob

rbeezer commented 4 years ago

Forgot to say. We have these optional "archive links" for images, which Alex put together. An author might want to provide a static 2D SVG of a 3D object, say, for an instructor to use on an exam or worksheet or ... Even if a 3D live version is in the online book.

sean-fitzpatrick commented 4 years ago

Right, I was thinking of those archive links as a case where we might want to run mbx with both svg and html output formats. If there's 3D content, that will mean running mbx with pdf as the outformat (maybe we want that too, anyway) and then converting pdf to svg.

rbeezer commented 4 years ago

Pushed 3 commits, ending with 787a2cc823a83b670a6861207aea33977d1acc37.

Support for a asymptote/@dimensions attribute with allowed values of 2 or 3. Needs to go into schema. Allows author to be explicit. No testing at all.
Extraction stylesheet produces Python list items with filename and dimension information. Authored attribute is checked first. Otherwise, a lame template decides, but at best I'd call this an educated guess. There are many ways to fool it. But with author education it'll behave nicely.
mbx script has been refactored to rely on upgraded info, but no new logic has been added. Search on diagram[1] to see a code comment that can be removed once employed.

Over to you, Sean. Remember one example in the sample article has graph3 in a comment line!

sean-fitzpatrick commented 4 years ago

It may be a week or two until I look at this. Teaching, and stuff. Survival mode until the mid-February break.

Maybe we can nudge @mdoob for an opinion again? My only concern is making sure that this doesn't complicate authoring and building. (I trust the @dimensions attribute is optional?)

I want to make sure I understand what use cases are addressed by this that can't be handled by just adding a bit more logic to mbx. (Unless we're concerned about runtime impacts from too many if/then/else bloating the code?)

mdoob commented 4 years ago

I guess my previous comment didn't go through. I'll try again.

My recollection is that running mbx with the -svg option called asymptote with the same option; alas, this didn't produce any output, and eventually mbx tried to read the nonexistent file and threw an error. That's why I went to -f pdf, which works properly, and then did the pdf to svg conversion myself.

Relying on the author to use some specific syntax seems error-prone to me. Is there any way to run asymptote in some kind of debug mode to determine if the graph3 package has been requested?

Cheers, Michael

sean-fitzpatrick commented 4 years ago

I think your previous comment landed on an issue and not on this pull request. :-)

Rob has built in some infrastructure to detect 2d vs 3d Asymptote (or to let authors specify).

As I understand it:

asy -f pdf works on all formats, but needs to be converted to svg for web use
asy -f svg works on 2d but fails on 3d.
the new option of asy -f html works on all formats, but outputs 2d to svg and 3d to html (WebGL).

The last is what I'd be using. But it requires version 2.62 (latest) for Asymptote. It'll fail on anything earlier than 2.61.

I think there is still one question outstanding: if someone chooses svg for the outformat in mbx, did they overlook the fact that this won't work for 3d graphs, or were they expecting mbx to produce pdf and then convert to svg?

In either case, do we need to detect the presence of graph3 on advance? Or if no svg is produced because asy failed as noted above, can we just fall back to the svg via pdf pathway?

mdoob commented 4 years ago

Would the following make sense, at least for the time being? When a user includes asymptote files within a document, use a test to see if pdfsvg or some equivalent is locally available, and, if so, replace the running of asymptote -svg by asymptote -pdf and then convert the resulting file to svg.

sean-fitzpatrick commented 4 years ago

Possibly. I think the reason to test for use of graph3 is that for 2D asymptote, we should expect that direct export to SVG from asy is going to be better than doing asy -f pdf followed by pdf2svg conversion. I think (and Rob can correct me if this is wrong) that the only usage of mbx with asymptote images that is potentially problematic is going to be running asy -v -c asy -f svg.

Everything works as it should for pdf as the outformat. (Or does now, after the work we did in the fall, to add the -noprc option and one or two other tweaks to the XSL.)

Everything works as it should for html as the outformat, as long as the asy binary is at 2.62 or newer. I believe Rob already added a warning in the mbx output about this, whenever html output is called.

Pretty sure the only scenario that needs work to support is when an author/publisher wants an SVG version of a 3D image. (This would be needed for any offline version -- perhaps ePub. But we might want SVG available alongside the interactive HTML.)

If I understand correctly there are two ways to accomplish this:

XSL reads the source, looking for graph3. If graph3 is detected in an asymptote component, mbx is instructed to run asy -f pdf, followed by pdf2svg.
We just run mbx, with svg as the outformat, and if an SVG image is not found (because asy -f svg failed in the background on a 3D graph) then mbx tries again with asy -f pdf)

As Rob pointed out, option 1 might detect some false positives. But even if there's a false positive, I think this just results in SVG being created via PDF, rather than directly. (Which is what we'd be doing with Michael's suggestion anyway.)

rbeezer commented 4 years ago

Yes, that is all pretty much exactly as I see it. And I'd like to start with (1) as the lesser of two evils. Making the graph3 guess realtively sure-fire requires just a small dose of author education. Since a failure to build any output can happen for a wider variety of reasons, I'd rather not rely on that as a signal.

Now if the Asymptote executable had some diagnostic tool that might report a list of modules in use, we could smarten the guess within mbx. Right now, the guessing is in the XSL stylesheet.

rbeezer commented 4 years ago

I don't think there is anything here left to merge, and that discussion and work has moved on to #1275. Feel free to request that this be re-opened for some reason.

sean-fitzpatrick commented 4 years ago

This seems right. The initial commits to add the html outformat, and check for svg if no html exists, are already merged. Now we just have to get the svg support right in #1275

PreTeXtBook / pretext

Add support for Asymptote WebGL #1238