search

daisy / pipeline-modules

Modules for the DAISY Pipeline project
3 stars 4 forks source link

dtbook-to-pef : script is crashing the pipeline #102

Open NPavie opened 1 month ago

NPavie commented 1 month ago

On windows 11, with the pipeline app embedded engine (from the UI and from the cli with the engine)

When using the 1.14.19-bugfix branch of the assembly (that references the 1.14.27-SNAPSHOT version of the module repo), the dtbook to braille script is crashing very often on the "Transforming from OBFL to PEF".

Note that when it's not crashing, relaunching the job does provoke the following error in the job log

2024-07-16 15:40:24,127 [DEBUG] o.d.p.b.d.c.impl.OBFLToPEFStep - Created CompoundBrailleTranslator{translators={uncontracted=CompoundBrailleTranslator{translators={auto=LiblouisTranslatorJnaImplProvider$LiblouisTranslatorImpl{translator=Translator{table=file:/C:/Users/npavie/AppData/Local/Temp/pipeline-16663242184350624789/spaces.cti,http://www.liblouis.org/tables/en-us-g1.ctb}, displayTable=file:/C:/Users/npavie/AppData/Local/Temp/pipeline-16663242184350624789/unicode.dis,file:/C:/Users/npavie/AppData/Local/Temp/pipeline-16663242184350624789/spaces.dis, hyphenator=LibhyphenJnaImpl$LibhyphenHyphenatorImpl{language=en, table=hyph_en_US.dic, id=transform9}, id=transform10}, none=UnityBrailleTranslator{id=transform7}}, id=transform14}, auto=CompoundBrailleTranslator{translators={auto=LiblouisTranslatorJnaImplProvider$LiblouisTranslatorImpl{translator=Translator{table=file:/C:/Users/npavie/AppData/Local/Temp/pipeline-16663242184350624789/spaces.cti,http://www.liblouis.org/tables/en_CA.tbl}, displayTable=file:/C:/Users/npavie/AppData/Local/Temp/pipeline-16663242184350624789/unicode.dis,file:/C:/Users/npavie/AppData/Local/Temp/pipeline-16663242184350624789/spaces.dis, hyphenator=LibhyphenJnaImpl$LibhyphenHyphenatorImpl{language=en, table=hyph_en_US.dic, id=transform9}, id=transform15}, none=UnityBrailleTranslator{id=transform2}}, id=transform17}}, id=transform20}
2024-07-16 15:40:24,127 [DEBUG] o.d.p.b.d.c.impl.OBFLToPEFStep - Created CounterHandlingBrailleTranslator{id=transform21}
2024-07-16 15:40:26,934 [INFO ] org.daisy.pipeline.job.Job - Changing job status to: ERROR
2024-07-16 15:40:26,935 [ERROR] org.daisy.pipeline.job.Job - job finished with error state
Unexpected error in pxi:obfl-to-pef
    at {http://www.daisy.org/ns/pipeline/xproc/internal}obfl-to-pef(obfl-to-pef.xpl:69)
    at {http://www.daisy.org/ns/pipeline/xproc}obfl-to-pef(dotify-transform.xpl:90)
    at {http://www.w3.org/ns/xproc}when(dotify-transform.xpl:89)
    at {http://www.w3.org/ns/xproc}choose(dotify-transform.xpl:88)
    at {http://www.daisy.org/ns/pipeline/xproc}transform(dtbook-to-pef.convert.xpl:265)
    at {http://www.w3.org/ns/xproc}otherwise(dtbook-to-pef.convert.xpl:254)
    at {http://www.w3.org/ns/xproc}choose name="transform"(dtbook-to-pef.convert.xpl:171)
    at {http://www.daisy.org/ns/pipeline/xproc}dtbook-to-pef name="convert"(__processed__dtbook-to-pef.xpl:307)
Caused by: Invalid memory access
    at com.sun.jna.Native.invokeInt(Native Method)
    at com.sun.jna.Function.invoke(Function.java:429)
    at com.sun.jna.Function.invoke(Function.java:364)
    at com.sun.jna.Library$Handler.invoke(Library.java:270)
    at jdk.proxy2/jdk.proxy2.$Proxy13.hnj_hyphen_hyphenate2(Unknown Source)
    at ch.sbs.jhyphen.Hyphenator.hyphenate(Hyphenator.java:134)
    at org.daisy.pipeline.braille.libhyphen.impl.LibhyphenJnaImpl$LibhyphenHyphenatorImpl$1.getHyphenationOpportunities(LibhyphenJnaImpl.java:267)
    at org.daisy.pipeline.braille.common.AbstractHyphenator$util$DefaultFullHyphenator.transform(AbstractHyphenator.java:331)
    at org.daisy.pipeline.braille.common.AbstractHyphenator$util$DefaultFullHyphenator.transform(AbstractHyphenator.java:239)
    at org.daisy.pipeline.braille.common.AbstractHyphenator$util$LanguageBasedDispatchingFullHyphenator.transform(AbstractHyphenator.java:83)
    at org.daisy.pipeline.braille.common.AbstractHyphenator$util$DefaultFullHyphenator.transform(AbstractHyphenator.java:138)
    at org.daisy.pipeline.braille.liblouis.impl.LiblouisTranslatorJnaImplProvider$HyphenatorAsFullHyphenator.transform(LiblouisTranslatorJnaImplProvider.java:1554)
    at org.daisy.pipeline.braille.liblouis.impl.LiblouisTranslatorJnaImplProvider$LiblouisTranslatorImpl.transform(LiblouisTranslatorJnaImplProvider.java:1052)
    at org.daisy.pipeline.braille.liblouis.impl.LiblouisTranslatorJnaImplProvider$LiblouisTranslatorImpl.access$600(LiblouisTranslatorJnaImplProvider.java:212)
    at org.daisy.pipeline.braille.liblouis.impl.LiblouisTranslatorJnaImplProvider$LiblouisTranslatorImpl$3.transform(LiblouisTranslatorJnaImplProvider.java:366)
    at org.daisy.pipeline.braille.common.BrailleTranslator$FromStyledTextToBraille.transform(BrailleTranslator.java:28)
    at org.daisy.pipeline.braille.liblouis.impl.LiblouisTranslatorJnaImplProvider$LiblouisTranslatorImpl$LineBreaker.translateAndHyphenate(LiblouisTranslatorJnaImplProvider.java:401)
    at org.daisy.pipeline.braille.common.AbstractBrailleTranslator$util$DefaultLineBreaker.transform(AbstractBrailleTranslator.java:316)
    at org.daisy.pipeline.braille.common.CompoundBrailleTranslator$2$1.transform(CompoundBrailleTranslator.java:256)
    at org.daisy.pipeline.braille.common.CompoundBrailleTranslator$TransformImpl.transform(CompoundBrailleTranslator.java:157)
    at org.daisy.pipeline.braille.common.CompoundBrailleTranslator$2.transform(CompoundBrailleTranslator.java:266)
    at org.daisy.pipeline.braille.common.CompoundBrailleTranslator$2$1.transform(CompoundBrailleTranslator.java:256)
    at org.daisy.pipeline.braille.common.CompoundBrailleTranslator$TransformImpl.transform(CompoundBrailleTranslator.java:157)
    at org.daisy.pipeline.braille.common.CompoundBrailleTranslator$2.transform(CompoundBrailleTranslator.java:266)
    at org.daisy.pipeline.braille.dotify.impl.CounterHandlingBrailleTranslator.transform(CounterHandlingBrailleTranslator.java:64)
    at org.daisy.pipeline.braille.dotify.impl.BrailleTranslatorFactoryServiceImpl$BrailleTranslatorFromBrailleTranslator.translate(BrailleTranslatorFactoryServiceImpl.java:172)
    at org.daisy.pipeline.braille.dotify.impl.BrailleTranslatorFactoryServiceImpl$BrailleTranslatorFromBrailleTranslator.translate(BrailleTranslatorFactoryServiceImpl.java:168)
    at org.daisy.dotify.formatter.impl.row.SegmentProcessor.toResult(SegmentProcessor.java:845)
    at org.daisy.dotify.formatter.impl.row.SegmentProcessor.layoutTextSegment(SegmentProcessor.java:603)
    at org.daisy.dotify.formatter.impl.row.SegmentProcessor.loadNextSegment(SegmentProcessor.java:533)
    at org.daisy.dotify.formatter.impl.row.SegmentProcessor.prepareNext(SegmentProcessor.java:459)
    at org.daisy.dotify.formatter.impl.row.SegmentProcessor.getNext(SegmentProcessor.java:503)
    at org.daisy.dotify.formatter.impl.row.SegmentProcessor.getNext(SegmentProcessor.java:477)
    at org.daisy.dotify.formatter.impl.row.BlockContentManager.hasNext(BlockContentManager.java:123)
    at org.daisy.dotify.formatter.impl.page.RowGroupProvider.hasNext(RowGroupProvider.java:96)
    at org.daisy.dotify.formatter.impl.page.BlockProcessor.hasNextInBlock(BlockProcessor.java:82)
    at org.daisy.dotify.formatter.impl.page.ScenarioData.processBlock(ScenarioData.java:81)
    at org.daisy.dotify.formatter.impl.page.ScenarioProcessor.processBlock(ScenarioProcessor.java:123)
    at org.daisy.dotify.formatter.impl.page.ScenarioProcessor.process(ScenarioProcessor.java:60)
    at org.daisy.dotify.formatter.impl.page.BlockSequence.selectScenario(BlockSequence.java:65)
    at org.daisy.dotify.formatter.impl.page.PageSequenceBuilder2.<init>(PageSequenceBuilder2.java:156)
    at org.daisy.dotify.formatter.impl.sheet.SheetDataSource.ensureBuffer(SheetDataSource.java:258)
    at org.daisy.dotify.formatter.impl.sheet.SheetDataSource.getRemaining(SheetDataSource.java:160)
    at org.daisy.dotify.formatter.impl.VolumeProvider.prepare(VolumeProvider.java:122)
    at org.daisy.dotify.formatter.impl.FormatterImpl.getVolumes(FormatterImpl.java:194)
    at org.daisy.dotify.formatter.impl.FormatterImpl.write(FormatterImpl.java:157)
    at org.daisy.dotify.formatter.impl.engine.LayoutEngineImpl.convert(LayoutEngineImpl.java:121)
    at org.daisy.pipeline.braille.dotify.calabash.impl.OBFLToPEFStep.run(OBFLToPEFStep.java:309)
    at {http://www.daisy.org/ns/pipeline/xproc/internal}obfl-to-pef(obfl-to-pef.xpl:69)
    ... 7 more

When debugging other attempts where the engine and jvm crashes without reporting errors, I followed the same traces (i got up from the run function of the step up to org.daisy.dotify.formatter.impl.sheet.SheetDataSource.ensureBuffer(SheetDataSource.java:258) but I stopped here)

bertfrees commented 1 month ago

FIxed by https://github.com/daisy/jhyphen/pull/8.