search

jason-fox / fox.jason.passthrough.pandoc

Pandoc DITA-OT Plug-in for extending the available input formats for DITA-OT. Non DITA input sources can be pre-processed to create create valid DITA source.
https://jason-fox.github.io/dita-ot-plugins/pandoc
Apache License 2.0
20 stars 4 forks source link

Plugin fails to find pandoc even though it is installed #8

Closed rachrobts closed 3 years ago

rachrobts commented 3 years ago

I receive this error when I run DITA-OT build, even though I've confirmed pandoc is installed and can run from anywhere: /Applications/Oxygen XML Editor/frameworks/dita/DITA-OT3.x/plugins/fox.jason.passthrough.pandoc-master/resource/antlib.xml:27: Execute failed: java.io.IOException: Cannot run program "pandoc" (in directory "/var/folders/hm/51jzs7b97tvctw489dj_9w180000gp/T"): error=2, No such file or directory

jason-fox commented 3 years ago

Can you run a sample in debug mode (e.g. dita -d -f html5 -i XXX -o YYY) and paste or attach your debug trace here.

jason-fox commented 3 years ago

From the current error message, it looks like your java.io.tmpdir has been deleted somehow. I can add a defensive mkdir to see if that helps. Please reinstall from master:

dita uninstall fox.jason.passthrough.pandoc
dita install https://github.com/jason-fox/fox.jason.passthrough.pandoc/archive/master.zip
rachrobts commented 3 years ago

terminal-dita-debug-log.txt oxygen-dita-log.txt

I've reinstalled DITA-OT and all plugins

jason-fox commented 3 years ago

oxygen-dita.log contains the following:

pandoc.check:
   [pandoc] Execute failed: java.io.IOException: Cannot run program "pandoc" 
   (in directory "/Applications/Oxygen XML Editor/frameworks/dita/DITA-OT3.x"): error=2, 
   No such file or directory
[PAND001W][WARN] Pandoc has not been installed

So pandoc hasn't been installed correctly or is not found on the classpath during that run.

terminal-dita-debug.log contains the following:

pandoc.check:
   [pandoc] Current OS is Mac OS X
   [pandoc] Output redirected to property: pandoc.out
   [pandoc] Executing 'pandoc' with arguments:
   [pandoc] '-v'
   [pandoc] 
   [pandoc] The ' characters around the executable and arguments are
   [pandoc] not part of the command.
   [pandoc] pandoc 2.13
   [pandoc] Compiled with pandoc-types 1.22, texmath 0.12.2, skylighting 0.10.5,
   [pandoc] citeproc 0.3.0.9, ipynb 0.1.0.1
   [pandoc] User data directory: /Users/rroberts/.local/share/pandoc
   [pandoc] Copyright (C) 2006-2021 John MacFarlane. Web:  https://pandoc.org
   [pandoc] This is free software; see the source for copying conditions. There is no
   [pandoc] warranty, not even for merchantability or fitness for a particular purpose.

So this run has found Pandoc - I assume this was after your reinstall.

The failure is now occurring here:

 [gen-list] Using Xerces grammar pool for DTD and schema caching.
 [gen-list] Processing file:/Users/rroberts/Downloads/fox%20test/output-html/document.ditamap
 [gen-list] Processing file:/Users/rroberts/Downloads/fox%20test/output-html/swagger.json
 [gen-list] 
 [gen-list] swagger.process:
 [gen-list]      [move] Moving 1 file to /var/folders/hm/51jzs7b97tvctw489dj_9w180000gp/T
 [gen-list] 
 [gen-list] BUILD SUCCESSFUL
 [gen-list] Total time: 0 seconds
 [gen-list] 
 [gen-list] pandoc.process:
 [gen-list]    [pandoc] Result: 22
 [gen-list]    [pandoc] [WARNING] Could not deduce format from file extension  or  or .lua
 [gen-list]    [pandoc]   Defaulting to markdown
 [gen-list]    [pandoc] Unknown output format /applications/oxygen

The *.json extension of swagger.json is not a recognized extension of Pandoc, so maybe your *.ditamap is referencing it incorrectly.

For formats like Word, RestructuredText, Markdown etc. set the format="pandoc" as shown.

<bookmap>
    <chapter format="pandoc" href="markdown.md" type="topic"/>
</bookmap>

From the name of the file, I think you may be attempting to generate a swagger definition. That is a separate plugin and requires format="swagger":

<bookmap>
    <chapter format="swagger" processing-role="normal" type="topic" href="swagger.json"/>
</bookmap>

The error Unknown output format /applications/oxygen is probably due to installing Pandoc into a folder with a space character within in. It appears to be picking up part of the path /Applications/Oxygen XML Editor/ - try reinstalling pandoc into /Applications/Pandoc or moving the dita installation.

jason-fox commented 3 years ago

I've added defensive quotes around the lua template parameter, maybe that will help. Please reinstall from master again:

dita uninstall fox.jason.passthrough.pandoc
dita install https://github.com/jason-fox/fox.jason.passthrough.pandoc/archive/master.zip
rachrobts commented 3 years ago

So pandoc hasn't been installed correctly or is not found on the classpath during that run.

I installed pandoc using the package here: https://github.com/jgm/pandoc/releases/tag/2.13

So this run has found Pandoc - I assume this was after your reinstall.

No, not after reinstall. One is what happens when I run from Oxygen Editor and the other is from terminal in debug mode. I contacted Radu at Oxygen first and he said they couldn't do much it's not their plugin.

From the name of the file, I think you may be attempting to generate a swagger definition. That is a separate plugin and requires format="swagger":

Yes I'm using the swagger plugin, which also has me install this plugin and this one seemed to be the one it was failing at. Maybe I'm commenting on the wrong project. I do have format="swagger" set. I'm actually using fox-test output-html (though I've tried my own ditamap too)

The error Unknown output format /applications/oxygen is probably due to installing Pandoc into a folder with a space character within in. It appears to be picking up part of the path /Applications/Oxygen XML Editor/ - try reinstalling pandoc into /Applications/Pandoc or moving the dita installation.

I used the pandoc installer pkg which puts it in /usr/local/bin. I then manually added that to $PATH. Dita is installed at /Applications/Oxygen XML Editor/frameworks/dita/DITA-OT3.x which I believe is the default directory for Oxygen Editor

Just reinstalled plugins, not sure why I'm having so much trouble getting this going

jason-fox commented 3 years ago

I have created a branch with more debug:

dita uninstall fox.jason.passthrough.pandoc
dita install https://github.com/jason-fox/fox.jason.passthrough.pandoc/archive/debug.zip

Please run the following:

dita -o ../out -i ../plugins/fox.jason.passthrough.pandoc/test/input-markdown/document.ditamap -f html5 -v

Two files should be created in dita root - run.txt and check.txt, they should look something like this:

CHECK

<<< TASK FINISHED -- org.apache.tools.ant.UnknownElement@40499e4f
>>> TASK STARTED -- org.apache.tools.ant.UnknownElement@51cd7ffc
--- MESSAGE LOGGED
   [pandoc] Current OS is Mac OS X
--- MESSAGE LOGGED
   [pandoc] Output redirected to property: pandoc.out
--- MESSAGE LOGGED
   [pandoc] Executing 'pandoc' with arguments:
'-v'

The ' characters around the executable and arguments are
not part of the command.
--- MESSAGE LOGGED
Execute:Java13CommandLauncher: Executing 'pandoc' with arguments:
'-v'

The ' characters around the executable and arguments are
not part of the command.
--- MESSAGE LOGGED
Setting project property: pandoc.result -> 0
--- MESSAGE LOGGED
Setting project property: pandoc.out -> pandoc 2.11.2
Compiled with pandoc-types 1.22, texmath 0.12.0.3, skylighting 0.10.0.3,
citeproc 0.2, ipynb 0.1.0.1
User data directory: /Users/jasonfox/.local/share/pandoc or /Users/jasonfox/.pandoc
Copyright (C) 2006-2020 John MacFarlane. Web:  https://pandoc.org
This is free software; see the source for copying conditions. There is no
warranty, not even for merchantability or fitness for a particular purpose.
<<< TASK FINISHED -- org.apache.tools.ant.UnknownElement@51cd7ffc
>>> TASK STARTED -- org.apache.tools.ant.UnknownElement@7a791b66
<<< TASK FINISHED -- org.apache.tools.ant.UnknownElement@7a791b66
>>> TASK STARTED -- org.apache.tools.ant.UnknownElement@6f2cb653
--- MESSAGE LOGGED
setting a recorder for name /Users/jasonfox/Workspace/dita/dita-ot-3.6.1/check.txt

RUN

<<< TASK FINISHED -- org.apache.tools.ant.UnknownElement@79c3f01f
>>> TASK STARTED -- org.apache.tools.ant.UnknownElement@6c2f1700
<<< TASK FINISHED -- org.apache.tools.ant.UnknownElement@6c2f1700
>>> TASK STARTED -- org.apache.tools.ant.UnknownElement@350b3a17
--- MESSAGE LOGGED
    [mkdir] Skipping /var/folders/79/f71dhgdd4nz6hyv8p173ypb40000gn/T because it already exists.
<<< TASK FINISHED -- org.apache.tools.ant.UnknownElement@350b3a17
>>> TASK STARTED -- org.apache.tools.ant.UnknownElement@38600b
--- MESSAGE LOGGED
   [pandoc] Current OS is Mac OS X
--- MESSAGE LOGGED
   [pandoc] Error redirected to property: pandoc.error
--- MESSAGE LOGGED
   [pandoc] Executing 'pandoc' with arguments:
'/var/folders/79/f71dhgdd4nz6hyv8p173ypb40000gn/T/src2141990518394211341.md'
'-t'
'/Users/jasonfox/Workspace/dita/dita-ot-3.6.1/plugins/fox.jason.passthrough.pandoc/resource/topic.lua'
'-o'
'/var/folders/79/f71dhgdd4nz6hyv8p173ypb40000gn/T/dest11673857287633246566.tmp'
'-M'
'title=markdown'

The ' characters around the executable and arguments are
not part of the command.
--- MESSAGE LOGGED
Execute:Java13CommandLauncher: Executing 'pandoc' with arguments:
'/var/folders/79/f71dhgdd4nz6hyv8p173ypb40000gn/T/src2141990518394211341.md'
'-t'
'/Users/jasonfox/Workspace/dita/dita-ot-3.6.1/plugins/fox.jason.passthrough.pandoc/resource/topic.lua'
'-o'
'/var/folders/79/f71dhgdd4nz6hyv8p173ypb40000gn/T/dest11673857287633246566.tmp'
'-M'
'title=markdown'

The ' characters around the executable and arguments are
not part of the command.
<<< TASK FINISHED -- org.apache.tools.ant.UnknownElement@38600b
>>> TASK STARTED -- org.apache.tools.ant.UnknownElement@1ea9f009
<<< TASK FINISHED -- org.apache.tools.ant.UnknownElement@1ea9f009
>>> TASK STARTED -- org.apache.tools.ant.UnknownElement@5d52e3ef
--- MESSAGE LOGGED
setting a recorder for name /Users/jasonfox/Workspace/dita/dita-ot-3.6.1/run.txt
jason-fox commented 3 years ago

Also from command line:

pandoc -v
pandoc 2.11.2
Compiled with pandoc-types 1.22, texmath 0.12.0.3, skylighting 0.10.0.3,
citeproc 0.2, ipynb 0.1.0.1
User data directory: /Users/jasonfox/.local/share/pandoc or /Users/jasonfox/.pandoc
Copyright (C) 2006-2020 John MacFarlane. Web:  https://pandoc.org
This is free software; see the source for copying conditions. There is no
warranty, not even for merchantability or fitness for a particular purpose

Also run (amend the path as necessary)

pandoc \
'/Users/jasonfox/Workspace/dita/dita-ot-3.6.1/plugins/fox.jason.passthrough.pandoc/test/input-markdown/markdown.md' \
-t \
'/Users/jasonfox/Workspace/dita/dita-ot-3.6.1/plugins/fox.jason.passthrough.pandoc/resource/topic.lua' \
-o \
'out.dita' \
-M title=markdown

and see if out.dita is created.

rachrobts commented 3 years ago

check.txt:

<<< TASK FINISHED -- org.apache.tools.ant.UnknownElement@1b75c2e3
>>> TASK STARTED -- org.apache.tools.ant.UnknownElement@1984b1f
--- MESSAGE LOGGED
   [pandoc] Current OS is Mac OS X
--- MESSAGE LOGGED
   [pandoc] Output redirected to property: pandoc.out
--- MESSAGE LOGGED
   [pandoc] Executing 'pandoc' with arguments:
'-v'

The ' characters around the executable and arguments are
not part of the command.
--- MESSAGE LOGGED
Execute:Java13CommandLauncher: Executing 'pandoc' with arguments:
'-v'

The ' characters around the executable and arguments are
not part of the command.
--- MESSAGE LOGGED
Setting project property: pandoc.result -> 0
--- MESSAGE LOGGED
Setting project property: pandoc.out -> pandoc 2.13
Compiled with pandoc-types 1.22, texmath 0.12.2, skylighting 0.10.5,
citeproc 0.3.0.9, ipynb 0.1.0.1
User data directory: /Users/rroberts/.local/share/pandoc
Copyright (C) 2006-2021 John MacFarlane. Web:  https://pandoc.org
This is free software; see the source for copying conditions. There is no
warranty, not even for merchantability or fitness for a particular purpose.
<<< TASK FINISHED -- org.apache.tools.ant.UnknownElement@1984b1f
>>> TASK STARTED -- org.apache.tools.ant.UnknownElement@e57b96d
<<< TASK FINISHED -- org.apache.tools.ant.UnknownElement@e57b96d
>>> TASK STARTED -- org.apache.tools.ant.UnknownElement@32c726ee
--- MESSAGE LOGGED
setting a recorder for name /Applications/OxygenXMLEditor/frameworks/dita/DITA-OT3.x/check.txt

run.txt:

<<< TASK FINISHED -- org.apache.tools.ant.UnknownElement@6d1d4d7
>>> TASK STARTED -- org.apache.tools.ant.UnknownElement@89ff02e
<<< TASK FINISHED -- org.apache.tools.ant.UnknownElement@89ff02e
>>> TASK STARTED -- org.apache.tools.ant.UnknownElement@6865c751
--- MESSAGE LOGGED
    [mkdir] Skipping /var/folders/hm/51jzs7b97tvctw489dj_9w180000gp/T because it already exists.
<<< TASK FINISHED -- org.apache.tools.ant.UnknownElement@6865c751
>>> TASK STARTED -- org.apache.tools.ant.UnknownElement@62679465
--- MESSAGE LOGGED
   [pandoc] Current OS is Mac OS X
--- MESSAGE LOGGED
   [pandoc] Error redirected to property: pandoc.error
--- MESSAGE LOGGED
   [pandoc] Executing 'pandoc' with arguments:
'/var/folders/hm/51jzs7b97tvctw489dj_9w180000gp/T/src7091944786590712391.md'
'-t'
'/Applications/OxygenXMLEditor/frameworks/dita/DITA-OT3.x/plugins/fox.jason.passthrough.pandoc/resource/topic.lua'
'-o'
'/var/folders/hm/51jzs7b97tvctw489dj_9w180000gp/T/dest6471306123163191093.tmp'
'-M'
'title=markdown'

The ' characters around the executable and arguments are
not part of the command.
--- MESSAGE LOGGED
Execute:Java13CommandLauncher: Executing 'pandoc' with arguments:
'/var/folders/hm/51jzs7b97tvctw489dj_9w180000gp/T/src7091944786590712391.md'
'-t'
'/Applications/OxygenXMLEditor/frameworks/dita/DITA-OT3.x/plugins/fox.jason.passthrough.pandoc/resource/topic.lua'
'-o'
'/var/folders/hm/51jzs7b97tvctw489dj_9w180000gp/T/dest6471306123163191093.tmp'
'-M'
'title=markdown'

The ' characters around the executable and arguments are
not part of the command.
<<< TASK FINISHED -- org.apache.tools.ant.UnknownElement@62679465
>>> TASK STARTED -- org.apache.tools.ant.UnknownElement@5b6813df
<<< TASK FINISHED -- org.apache.tools.ant.UnknownElement@5b6813df
>>> TASK STARTED -- org.apache.tools.ant.UnknownElement@5f2606b
--- MESSAGE LOGGED
setting a recorder for name /Applications/OxygenXMLEditor/frameworks/dita/DITA-OT3.x/run.txt

pandoc -v

pandoc 2.13
Compiled with pandoc-types 1.22, texmath 0.12.2, skylighting 0.10.5,
citeproc 0.3.0.9, ipynb 0.1.0.1
User data directory: /Users/rroberts/.local/share/pandoc
Copyright (C) 2006-2021 John MacFarlane. Web:  https://pandoc.org
This is free software; see the source for copying conditions. There is no
warranty, not even for merchantability or fitness for a particular purpose.
Fetchs-MacBook-Pro-3:DITA-OT3.x rroberts$

out.dita is created

rachrobts commented 3 years ago

It's working now when I run dita from terminal!! But still not from Oxygen, I'll email them again terminal.txt oxygen.txt

jason-fox commented 3 years ago

This is the first problem line - checking to see if Pandoc is installed:

<exec executable="pandoc" dir="${dita.dir}" osfamily="unix" taskname="pandoc"
    resultproperty="pandoc.result" outputproperty="pandoc.out" failonerror="false" 
        failifexecutionfails="false">
    <arg line="-v"/>
</exec>

It appears that the terminal user has rights to run from the ${dita.dir}, but when running via Oxygen the user changes and the application does not have sufficient rights.

Options you could try:

a) Remove dir="${dita.dir}" altogether - according to ANT exec this defaults to the current working directory. b) Replace with dir="${java.io.tmpdir}" - this is the temporary directory used by the Java Virtual Machine (JVM) to create and store temporary files.

Pandoc is called in two places resource/antlib.xml and process_pandoc.xml - you may need to change both.

jason-fox commented 3 years ago

Another option - cd to your dita.dir(/Applications/OxygenXMLEditor/frameworks/dita/DITA-OT3.x ) and check the access rights. You can ensure that all users have execute rights by running:

chmod +x  /Applications/OxygenXMLEditor/frameworks/dita/DITA-OT3.x
rachrobts commented 3 years ago

Oxygen gave me this workaround that works:

One thing I discovered, at least on my side is that if on Mac OS you start Oxygen from the Terminal using "sh oxygen.sh" in the Oxygen installation folder, when Oxygen runs the DITA OT, the build file manages to run the "pandoc" executable. Otherwise if I start Oxygen by double clicking it's shortcut in the Finder, this does not work, it works only if the path to the pandoc executable "/usr/local/bin/pandoc" is fully specified in the build file. I do not yet know why this is the case. If I find a solution for this I will update you.

jason-fox commented 3 years ago

Useful to know - I've added a paragraph to the README.md a651475

jason-fox commented 3 years ago

@rachrobts Try reinstalling from master

dita uninstall fox.jason.passthrough.pandoc
dita install https://github.com/jason-fox/fox.jason.passthrough.pandoc/archive/master.zip

There is now a new cfg/configuration.properties file which can set the Pandoc path if necessary.

Uncomment the line here and amend as necessary