jason-fox / fox.jason.prismjs

An integration of PrismJS into the DITA Open Toolkit engine, enabling static HTML and PDF syntax highlighting.
https://jason-fox.github.io/dita-ot-plugins/prismjs
Apache License 2.0
5 stars 3 forks source link

java.lang.StackOverflowError #14

Closed logronio closed 1 year ago

logronio commented 2 years ago

Processing too many files will throw java.lang.StackOverflowError. Is there a limit to the number of files?

Error log snippet: Error: The following error occurred while executing this line: D:\Ixiasoft\OutputGenerator\prod-01\data\dita-ot-3.6\plugins\fox.jason.prismjs\process_prismjs.xml:169: The following error occurred while executing this line: D:\Ixiasoft\OutputGenerator\prod-01\data\dita-ot-3.6\plugins\fox.jason.prismjs\process_prismjs.xml:170: The following error occurred while executing this line: D:\Ixiasoft\OutputGenerator\prod-01\data\dita-ot-3.6\plugins\fox.jason.prismjs\resource\antlib.xml:307: java.lang.StackOverflowError

jason-fox commented 2 years ago

That is a strange line to be failing on. As a first attempt at a workaround, I would suggest increasing the memory allocated to the JVM as described here . My guess is that because <macrodef name="iterate-ditafiles> is recursive, the memory assigned to <local> elements inside it isn't being freed until all files have been processed.

You could also run with full debug using the -d parameter and post a larger chunk of debug. The error message itself isn't very useful for diagnosis without more context.

Another alternative would be to split the highlight target into more blocks, you could do this by reducing the name regex to cover a subset of directories e.g. ${prism.temp.dirname}/[A-F].* , ${prism.temp.dirname}/[G-M].* etc. and call iterate-ditafiles with each subgroup.

<target name="highlight" if="prismjs.highlightable">
    <basename property="prism.temp.dirname" file="${dita.temp.dir}"/>
    <restrict id="prismjs.ditafileset">
      <ditafileset processingRole="normal">
        <excludes format="ditamap"/>
        <excludes format="ditaval"/>
        <excludes format="image"/>
      </ditafileset>
      <and>
        <or>
          <contains text="&lt;codeph"/>
          <contains text="&lt;codeblock"/>
        </or>
        <name regex="\/${prism.temp.dirname}\/" handledirsep="true"/>
      </and>
    </restrict>
    <pathconvert
      property="prismjs.ditafiles"
      refid="prismjs.ditafileset"
      pathsep=";"
    >
      <map from="file:" to=""/>
    </pathconvert>

    <iterate-ditafiles
      list="${prismjs.ditafiles}"
      unless:blank="${prismjs.ditafiles}"
    >
      <highlight-dita-file file="${element}"/>
    </iterate-ditafiles>
  </target>

How many files are you processing? I have tried larger documents, but not really massive ones.

logronio commented 2 years ago

My estimate is at least 1000 files. I will try your suggested alternative solution.

jason-fox commented 2 years ago

I have switched the recursive function for a custom ANT task. Please reinstall and see if this helps to free up memory

dita uninstall fox.jason.prismjs
dita install https://github.com/jason-fox/fox.jason.prismjs/archive/master.zip
hcw70 commented 1 year ago

I had a similair mem overflow issue with this plugin. After update, it is resolved. So at least my issue is fixed with this.