search

TEIC / TEI

The Text Encoding Initiative Guidelines
https://www.tei-c.org
Other
274 stars 88 forks source link

Jar file setup for antruntest.xml needs updating #1547

Open martindholmes opened 7 years ago

martindholmes commented 7 years ago

P5/Test/antruntest.xml references a specific version of Saxon in the ../Utilities folder; that version should be replaced by a generic saxon9he.jar which can be updated without changing files that reference it.

Trang is pointed to (line 33) in /usr/share/trang; it is not in that location on my system, and there is no reason not to put trang.jar in the ../Utilities folder with Saxon and jing, and point to it there.

In other words:

  1. Replace Saxon 9.4 jar in Utilities with current saxon9he.jar.
  2. Add trang.jar to Utilities.
  3. Change lines 32 and 33 to:
  <property name="saxon" value="../Utilities/lib/saxon9he.jar"/>
  <property name="trang" value="../Utilities/trang.jar"/>

Other build files should be examined for similar oddities.

martindholmes commented 7 years ago

The Simple repo has lib/saxon9he.jar, which is Saxon-HE 9.6.0.5J. The Stylesheets repo has lib/saxon9he.jar, which is Saxon-HE 9.5.1.5J. The TEI repo has P5/Utilities/saxon9he.jar, which is Saxon-HE 9.5.1.5J, in addition to the Saxon-HE-9.4.0.6.jar. The OxGarage repo has saxon9he.jar in the root, but it appears to be broken.

lb42 commented 7 years ago

On my system, the Stylesheets lib/saxon9he.jar appears to provide saxon 9.7, so I copied that to Utilities/lib. I seem to have copies of trang.jar all over the place, including one in Utilities/lib so I changed the property in P5/Test/antruntest.xml to point to that, rather than making yet another. However that version of trang seem to go back to 2003, so I replaced it with the most recent version I could find from https://storage.googleapis.com/google-code-archive-downloads/v2/code.google.com/jing-trang/trang-20091111.zip : I then ran make clean; make test ... and errors started to appear, as predicted.

Here's the first one: /home/lou/Public/TEI/P5/antbuilder.xml:39: Fatal error during transformation using /home/lou/Public/TEI/P5/Utilities/expand.xsl: Base URI {} is not an absolute URI; SystemID: file:/home/lou/Public/TEI/P5/Utilities/expand.xsl; Line#: 83; Column#: 16 The line in question reads

  <xsl:variable name="info" select="doc(resolve-uri('../repodate.xml',base-uri(/)))"/>

Any suggestions?

hcayless commented 7 years ago

Trying this out with Saxon 9.7.14 and jing/trang V20131210

hcayless commented 7 years ago

Pushed changes. If the build succeeds, then I'll close this.

hcayless commented 7 years ago

Also removed extra copies of Jing and Saxon in Utilities/lib, but it turns out the Epub check has a dependency on a couple of them, so I've put them back. The Epub validator we use is incompatible with the latest version of Saxon because they change their API every new moon, so we can't just use it with the new Saxon. We should maybe consider upgrading the Epub checker to v4 from v3 though.

lb42 commented 7 years ago

Would clearly be a good idea to get latest versions of all dependencies.

hcayless commented 7 years ago

V4.01 of the Epub checker gives me a whole lot of warnings. Mostly it would like us to use .xhtml instead of .html as a file extension

martindholmes commented 7 years ago

Warnings we think are unnecessary we can allow for in the Jenkins log parser.

While we're on epub: the Makefile has this:

epub: Guidelines.epub

Guidelines.epub: check.stamp p5.xml
    @echo BUILD: Make epub version of Guidelines
    ${ANT} -f ${XSL}/epub3/build-to.xml -lib Utilities/lib/${SAXONJAR} -Dprofiledir=${XSL}/profiles -Dprofile=tei -DinputFile=`pwd`/p5.xml -DoutputFile=`pwd`/Guidelines.epub -Dcoverimage=`pwd`/Utilities/cover.jpg
    java -jar Utilities/epubcheck3.jar Guidelines.epub
    touch Guidelines.epub

epub3: check.stamp p5.xml
    ${ANT}  -f ${XSL}/epub3/build-to.xml -lib Utilities/lib/${SAXONJAR} -Dprofiledir=${XSL}/profiles -Dprofile=tei -DinputFile=`pwd`/p5.xml -DoutputFile=`pwd`/Guidelines.epub -Dcoverimage=`pwd`/Utilities/cover.jpg
    java -jar Utilities/epubcheck3.jar Guidelines.epub

It seems to me that both the epub and epub3 targets do the same thing: they build using the epub3 code. Anyone see any reason not to simplify this?

lb42 commented 7 years ago

With latest versions of TEI and Stylesheets from dev, I am able to run "make" to completion, modulo a lot of complaining about onvdl in the middle.

@martindholmes I conjecture that the epub3 target was added when checking against epub3 was new and we were checking against some previous version by default. we could change it to check against epub4, and maybe change the target name to epub-bleeding-edge vel sim.

martindholmes commented 7 years ago

I've looked at the latest epubcheck version 4, and its accompanying libs would need to be added to our Utilities/lib folder; they include newer versions of some libs that are already there (commons-compress, guava...) and also they annoyingly include an old version of Saxon (but newer than the one that's there now). It's possible that updating to epubcheck version 4 will introduce some incompatibilities with other jars, so it'll have to be done carefully.

@lb42 I think you're right about the epub/epub3 thing. There's a lot of this sort of stuff that needs carefully cleaning up.

lb42 commented 7 years ago

Ah, so my advice would be to change the Makefile to make it possible to test epub4 by explicitly doing a "make epub4", but to stick with epub3 as the default. At least until someone has had time to investigate all those dependencies more carefully.

martindholmes commented 7 years ago

We can only use the epubcheck 4 by overwriting some existing jars in Utilities/lib with newer versions, and that may break other stuff. I still think we have to do it, but given all the other stuff that's flaky I'm not sure this is the time for it.

hcayless commented 7 years ago

I think those are only there to support epubcheck.jar

On Mon, Dec 5, 2016 at 12:12 PM, Martin Holmes notifications@github.com wrote:

We can only use the epubcheck 4 by overwriting some existing jars in Utilities/lib with newer versions, and that may break other stuff. I still think we have to do it, but given all the other stuff that's flaky I'm not sure this is the time for it.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/TEIC/TEI/issues/1547#issuecomment-264914170, or mute the thread https://github.com/notifications/unsubscribe-auth/AABbxaWZis5Mc35LsjCP5g0QTo2jwy8Wks5rFEXlgaJpZM4LCuVt .

martindholmes commented 7 years ago

@hcayless: The actual kickoff for this ticket is the fact that antruntest.xml explicitly references the Saxon file in Utilities/lib, so even if they originally came in to support epubcheck, we know they've been referenced from elsewhere at least once.

hcayless commented 7 years ago

@martindholmes I wasn't talking about Saxon and Jing--those definitely get used elsewhere. But for Saxon at least, I think there's a hard dependency on a particular version in Utilities/lib for epubcheck.

martindholmes commented 7 years ago

We should consider another approach here. The run-onvdl shell script is called for oNVDL validation; that goes and gets a copy of nvdl and unzips it before using it, assuming it's not already there. Why not do this with epubcheck? It would mean we could remove all those files in the Utilities/lib folder, and we wouldn't have to worry about them getting out of date.

hcayless commented 7 years ago

Well, we'd still be pointing at a particular release, so we'd have the same sort of staleness problem, and there'd be the extra time to download everything, and you couldn't run the build offline.

In theory, since it's a Maven project, you could clone the repo, build it yourself, and execute the target jar. In practice that might be more work than we'd want to do for every build. We could point at their master branch, but we'd probably occasionally see failures due to bugs. Possibly we could work out a compromise solution where we'd build a particular tag, and only rebuild if the tag was changed...

martindholmes commented 7 years ago

It might be possible to figure out the latest tag:

https://github.com/IDPF/epubcheck/releases/latest

and then derive the download location. I wish I knew why Sebastian did onvdl that way; my guess is that it's something to do with not wanting to burden the TEI repo with all those extra jars, as well as the requirement to distribute the license info prominently.

martinascholger commented 4 years ago

@hcayless, @martindholmes -- is this still an open issue?

martindholmes commented 4 years ago

I think there's still work to be done, but perhaps we should split this out into one ticket for moving to saxon 10 and xslt3, and another for figuring out what to do about epubcheck.

hcayless commented 3 years ago

Looking at this again. I will try out the latest version of EPubCheck (https://github.com/w3c/epubcheck) and see where that leaves us