Using MPP with a different processor? (not markdown-it or pandoc)

[tom-newhall]

Hi,

Thank you in advance for reading this and taking the time to help me out.

I would really like to use MPP with a different Markdown processor, specifically one called Madoko. I know this is going beyond the standard functionality of the software, but if I could find a solution for this, it would really be a lifesaver.

I tried simply putting madoko (i.e. instead of pandoc) in the Path to Pandoc executable box, and keeping the Filters and Commandline Arguments boxes blank. This seems to get Madoko to run, but I get the following error message in the MPP preview pane:

unrecognized option "--to=html" unrecognized option "--from= " usage: madoko [options] files options: --version Display version information -v --verbose Be more verbose --odir=DIR Write output files to the specified directory --xmp Only process markdown between
 tags
              --tex              Generate a LaTeX file too
              --pdf              Generate PDF using LaTeX
              --texzip           Generate TeX zip for submission
              --png              Use PNG instead of SVG for math in HTML
              --convert-tex      Convert input from TeX to Markdown
 -f           --fragment         Generate a fragment instead of a full document
              --logo             Generate a logo at the end of the document
              --sanitize         Always escape or suppress user defined html
              --sandbox          Run in a sandbox for secure server execution
              --pedantic         Pedantic mode
              --bench            For benchmarking: turn off numbering, etc.
              --installdir=DIR   Set installation directory explicitly
 -r           --rebuild          Force rebuild bibliography, math, etc.
              --prelude=FILE     Include <FILE> at start of the document
              --verbose-max=LEN  Maximum line length for messages
 -m<key:val>  --meta=key:val     Semi-colon separated list of metadata values

Prefix a flag with 'no-' to negate it. For example '--no-logo'.

This seems to indicate that the madoko program is being invoked properly, but seems to be adding pandoc flags that I don't need.

I guess my questions are:

1. How can I get rid of those pandoc flags so I can use the madoko program instead?
2. How can I get MPP to get process the file I am working on in Atom with the madoko program instead of pandoc or markdown-it?
3. How can I then display the rendered html, which is by default saved as a separate .html file in a sub-folder (\out) (i.e. not to stdout, like pandoc) in the preview pane?

I know this is asking a lot, but thanks so much for your help

[tom-newhall]

For what it's worth, I am transitioning to use Pandoc mainly instead of Madoko, but there are some features of Madoko that I like, and beyond that, have a few projects that I have already started using Madoko that I would like to keep working on as-is.

Madoko does actually provide its own previewing software, which is good for most tasks, but it's a little buggy, in particular when it comes to scrolling. I am hoping that MPP does a better job and is more flexible!

[lierdakil]

Hi. Sorry for the late reply.

in particular when it comes to scrolling

The bad news is that if by "scrolling" you mean "scroll position synchronization between source and preview" then MPP isn't that great with that when using pandoc (good enough with markdown-it though).

The good news is, if you're on Linux or macOS, you can create a shell script that would wrap madoko invocation and give that to MPP in place of pandoc executable. The script would need to a) invoke madoko, and b) output the produced html file, something along the lines of

#!/bin/bash
madoko && cat /path/to/out/file.html

If Madoko doesn't read stdin, you would also need to pass stdin through a temp. file:

#!/bin/bash
temp=$(mktemp tmp.XXXXXXXXXX.md)
cat > "$temp"
madoko "$temp" && cat /path/to/out/file.html
rm "$temp"

(Disclaimer: I didn't test this)

If you're on Windows, I'm honestly not sure if the same could be achieved with a bat file or a PowerShell script (didn't really use Windows since maybe 2008, so I'm very rusty on what can work there). But I guess it wouldn't hurt to try?

[tom-newhall]

Thanks so much for your reply! Actually, after searching and searching I finally figured this out on my own, and almost got it to work, but I am still having some issues. Here's the script I used:

#!/bin/sh

# First, processes the file using the `madoko` program 
# writes the output it to the 'out' directory
# but the full path is hard-coded for now:

madoko --odir=out "/Users/Shared/Dropbox/1 Daoxuan; Vinaya School Research/1 四分律行事鈔/序 Preface (Study group with Alex and Xingyi)/序 Preface (Study group with Alex and Xingyi).mdk"

# Then, reads the .html file (in the ./out directory) and sends it to stdout using the `cat` command
# again, the path to the file is hard-coded for now

cat "/Users/Shared/Dropbox/1 Daoxuan; Vinaya School Research/1 四分律行事鈔/序 Preface (Study group with Alex and Xingyi)/out/序 Preface (Study group with Alex and Xingyi).html"

This solution is so close to working, its very exciting, but there are a few problems: Madoko doesn't really take the input as stdin; just the filename, so:

1. I'm not quite sure how to 'send' the file path/file name from atom to this script. I assume this can easily be done with flags or variables or something like that but I'm a noob.

2. While I was able to get the above script to:

   • a. Run the madoko program and

   • b. send the resulting .html file to MPP with the cat command, for whatever reason

   • If I run both the madoko command and the cat command as part of the same shell file, MPP prints the following error:

Error: Command failed: /Users/thomasnewhall/madoko_script_for_atom.sh --from= --to=html cat: stdout: Resource temporarily unavailable

That appears as a red 'fire' banner in the upper right corner. In the preview window I also get the message

Previewing Markdown Failed
Command failed: /Users/thomasnewhall/madoko_script_for_atom.sh --from= --to=html cat: stdout: Resource temporarily unavailable

I thought this was because perhaps the cat program was trying to read the .html output of the madoko program before it had been fully written to the hard drive or something, (and was thus 'temporarily unavailable') so I tried inserting both the wait command and the sleep 10s command between the madoko command and the cat command, but that didn't work.

Googling this error... I couldn't make heads or tails out of it, so any help figuring out what may be causing this would be great.

Anyway, basically I was able to get MPP to process my file on a change, and then get it to render it in MPP, but not do both in one smooth motion, yet.

As to the issue with scrolling, basically it looks like your program should work for me. The issue with the viewer that comes with madoko (on madoko.net, or with the madoko-local program) is that there's no real way to disable the automatic scroll, so it always jumps around when it refreshes. But I can do that with MPP so I think it should work for me.

Thanks so much again!

[tom-newhall]

It may have something to do with this issue: https://github.com/travis-ci/travis-ci/issues/4704 but I tried using the madoko program and then run cat with a much smaller file and it still had the same issue, so I'm not sure file size is the problem, but I could be wrong.

[lierdakil]

Here's a wrapper script: https://gist.github.com/lierdakil/1b73eed72ef8fea7085d409fe9b7746d

I've tested on Linux, and it seems to work with a little caveat (see below). Can't test on macOS though, so YMMV.

The caveat is madoko makes code blocks somewhat differently from pandoc/markdown-it, so MPP is ill-equipped to handle those and will error out. You can enable the following toggle in MPP settings to work around this issue:

[tom-newhall]

Hey, thanks so much again for getting back to me. I copied your script and tried to run it, but no luck. I got basically the same error as before. I thought perhaps MPP was interacting poorly with another plugin, so I disabled all my non-core plugins, but that didn't seem to fix the problem.

The below shows up in a red pop-up window in atom when I run the script:

Error: spawn /Users/thomasnewhall/madoko_script_for_MMP.sh ENOENT

Error reported when Markdown Preview Plus view is loading

spawn /Users/thomasnewhall/madoko_script_for_MMP.sh ENOENT
Hide Stack Trace
Error: spawn /Users/thomasnewhall/madoko_script_for_MMP.sh ENOENT
    at Process.ChildProcess._handle.onexit (internal/child_process.js:246:19)
    at onErrorNT (internal/child_process.js:429:16)
    at processTicksAndRejections (internal/process/task_queues.js:81:17)
This is likely a bug in Atom. This issue has already been reported.

Furthermore, if I continue to edit my Markdown, I get the following message in the preview pane:

Previewing Markdown Failed
spawn /Users/thomasnewhall/madoko_script_for_MMP.sh ENOENT

This... looks complicated, so if it's asking too much of you or MPP to solve then please say as much.

[lierdakil]

Uh. Double-check your paths and if everything is executable (chmod +x filename). ENOENT means "no such file or directory".

[lierdakil]

Okay, so I've had some time to play around, and I managed to reproduce 'Resource temporarily unavailable' error. And after some debugging, I believe I know why: NodeJS sets child_process's stdout to non-blocking mode, while most simple tools expect stdout to be in blocking mode. It boils down to output being produced faster than Node can consume it, hence the "temporarily unavailable".

Anywho, to combat this issue, I rewrote https://gist.github.com/lierdakil/1b73eed72ef8fea7085d409fe9b7746d in Pyhton (which makes dealing with this a little easier). The script requires Python >= 3.5. As an added bonus, Python hopefully has less of cross-platform weirdness than Bash & friends, and doesn't need to rely on external programs like cat.

[tom-newhall]

OMG, this works perfectly! Thank you so much! Yeah I thought that maybe had something to do with the program spitting out code faster than the plugin can consume it, but had no idea how to fix it.

Let me know if there's something I can do for you in return...