search

eerohele / saxon-gradle

A Gradle plugin for running XSLT transformations with Saxon
MIT License
16 stars 6 forks source link

Parallel execution causes random I/O errors #28

Open ndw opened 3 years ago

ndw commented 3 years ago

I know this is a kind of lame bug report, but I'm afraid I haven't been able to narrow it down.

If you run a lot of XSLT transformations in parallel with the plugin, Gradle sometimes causes the plugin to throw I/O errors. I don't think this is related to Saxon, I think it's an issue with the WorkQueue or Gradle somehow. I say this because I've written two other plugins based on the model of saxon-gradle and they exhibit the same behavior even when Saxon isn't involved.

With the Saxon plugin, this is usually surfaces as a java.lang.ExceptionInInitializerError exception with no further detail. I think this is caused when Saxon is initializing and cannot load its configuration files. (Language information for regexp character classes and the like.)

I've "fixed" this by adding an option that allows the user to disable parallel execution. I'll see about constructing a PR for this when I get a chance.

neilccbrown commented 2 years ago

In case it helps anyone else in the future: I believe I encountered this exact same problem. I filed it as a bug with the full stack trace against Saxon-HE: https://saxonica.plan.io/issues/5685 With the help of Michael Kay there, I tracked the issue down to the fact that saxon-gradle runs each work task using classLoaderIsolation() which it seems is something that SaxonHE doesn't support. I had forked saxon-gradle anyway before that point (to get XInclude working correctly) but the fix (as detailed in a comment on that bug) was to change classLoaderIsolation() to noIsolation() and everything seems to work fine now.

eerohele commented 2 years ago

Thanks for reporting the finding! It's been a while, so the details elude me, but I believe the choice to use classLoaderIsolation() might have been motivated by letting the user choose the Saxon version to use. I could be wrong, though.

If you're able to determine that switching to noIsolation() causes no issues (that is, the current set of tests pass) and you can still choose to use other Saxon versions, I'd accept a PR with that change.

ndw commented 2 years ago

The interesting thing is that I have a fork of this repo (alas, probably too idiosyncratically divergent now) that I've been using for ages and I've never seen this issue. Do you have a sense of how your build.gradle file happens to be causing it?

neilccbrown commented 2 years ago

@ndw: Here is my Gradle build file (although be aware I've forked saxon-gradle into my repository so SaxonXsltTask is my task -- the main changes I made compared to eerehole's version were to fiddle with XInclude processing, although I also removed some code I didn't need): https://github.com/neilccbrown/columnal/blob/main/xml/build.gradle

I don't think it's (only) an issue with parallel gradle tasks because I made them sequential (see end of the Gradle file, although I think those bug links may be misleading now we've found the real cause). So it's somehow that previous sequential tasks with another classloader are interfering with later tasks -- they are all run directly one after the other. If you want to dig deeper you can see all the XSLT files here: https://github.com/neilccbrown/columnal/tree/main/xml/src/main/xslt