Why aren't all pandoc command line options supported

[denismaier]

I have just tried converting a document using the relatively new -d option, which resulted in codebraid throwing errors... Why? Aren't pandoc arguments just passed through?

[denismaier]

To add to this: I don't know how codebraid works, but could it be used as a pandoc filter? That way codebraid wouldn't have to deal with pandoc options at all.

[gpoore]

I'll look into adding support for the new -d option. This may take some time, since Codebraid will now have to emulate Pandoc's merging of command-line options with options in default files. Pandoc arguments aren't just passed through because documents must be processed in multiple steps. For the same reason, Codebraid wouldn't work as a filter.

Codebraid sends a document through Pandoc multiple times. It uses Pandoc to convert a document into AST form, then replaces code with raw output in the AST, then uses Pandoc to convert the AST back to Markdown, and finally uses Pandoc to convert this output Markdown to the final format. This guarantees that when Codebraid executes code to generate Markdown text, that Markdown text will be interpreted by Pandoc within the full context of the document. Running separate Pandoc processes on generated Markdown snippets would be simpler, but it would lose context information (for example, references wouldn't work in some cases). All user-supplied Pandoc options are only guaranteed to be used the final time that Pandoc runs. Some options must be added/removed in the intermediate steps to make everything work correctly.

[denismaier]

I understand. Would be great if this could be added, but no need to hurry.

In the meantime, would using codebraid as a kind-of-preprocessor be a workaround? Like running codebraid pandoc -t json to produce a temporary file. Then, this file could be transformed with pandoc using all the regular options. What do you think? (If this works, using pipes would also be an option on systems that support this.)

[gpoore]

Using codebraid as a preprocessor should work. I've tried to design things so that converting from Markdown to Markdown gives essentially the original document except with code replaced by output. If you find a case where that doesn't work, it's a bug. So you could try converting to a Markdown temporary file, and then running additional conversions after that. Converting to JSON instead of to Markdown isn't currently more efficient, because I haven't optimized the intermediate conversions to skip steps that aren't needed based on the output format.

[denismaier]

Ok. What exactly do you mean with markdown to markdown producing the original document with code replaced by output? What about setex vs atx style headers? When I thought about using a json temporary file my concern was not so much performance but rather such options where one choice could lead to a lossy transfoelation.

[denismaier]

Ah... transformation

[gpoore]

Running something like codebraid pandoc -f markdown -t markdown -o converted.md file.md will create essentially a copy of file.md with code replaced by output. Details like header style may change, but the transformation shouldn't lose any information. Whether you go straight to a final format like HTML, or to an intermediate Markdown file first, you should always get the same final output.

[gpoore]

I've added support for --defaults in the dev version, and this will be in a new version on PyPI soon.

Currently, there is a limitation that the input format, output format, and output file must be given on the command line when using --defaults (so they can't be specified in the defaults file). I may be able to relax that in the future, but doing so would probably require that codebraid parse all the defaults files and handle defaults merging in a Pandoc-equivalent manner. The current approach basically just requires explicit command-line options for the few pieces of information that codebraid requires.