New builder

[cchalmers]

Work in progress — don’t merge.

Changes related to improving diagrams-doc: https://groups.google.com/forum/#!topic/diagrams-discuss/LQvPrTfMvng

I’ve been playing around with builder and pandoc and had some ideas. The ticked boxes are things that are at least partially implemented in the PRs. I’d like to hear thoughts/suggestions.

lib changes

• [x] BackendBuild class with

  outputSize :: Lens' (Options b v n) (SizeSpec V2 n)
  
  saveDia :: FilePath -> Options b v n -> QDiagram b v n Any -> IO (Maybe String) 

• [x] "smarter" sandbox finder
• [x] clean up CmdLine loop output

  builder changes

Still not entirely happy with how hashing/results are returned.

• [x] More versatile — can return diagrams, results or save result to file using BackendBuild class
• [x] Hashing is done automatically for backends with BackendBuild.
• [ ] executables for building / watching a file

  pandoc changes

This still needs a lot of work but the basic idea is there.

• [x] support for multiple backends in same markdown file
• [x] options for width, height etc. from code block attributes for specific diagrams
• [x] backends can adjust for different output formats
• [x] executable for jsonfiltering and library for more complicated uses
• [x] images are stored with hash name and document references these hashed files (no other way to do it yet)
• [x] backend filters have access to document Meta (the YAML at the top of the md file).
• [ ] snippets support (define a snippet in the document to be included in the diagrams)

  doc changes

I think doc should ditch Cairo in favour of Rasterific, SVG and PGF (for PDFs) so we can all build the same documentation.

• [x] Replace rst with markdown. Pandoc has an RTS reader but it doesn’t seem to do everything properly yet. I’m pretty sure Pandoc’s markdown can support all the features we use in rst.
• [ ] pdf version of documentation (in progress)
• [ ] linkify functions and modules (CodeBlocks not linkified yet but single words are linking properly)
• [ ] get working with Shake and the site

[bergey]

As far as I know, pandoc HEAD has all the RST features we use. In particular, see https://github.com/jgm/pandoc/pull/1781 and https://github.com/jgm/pandoc/pull/1805

I think that also means we can use pandoc to convert .rst to .md if we want. I don't have a strong preference, but I think it would be nice to have all the docs in one format.

I'm not sure how much work it is to convert Xml2Html to a pandoc filter, but that's probably my next step.

[bergey]

On a quick read, the rest of this PR looks good. I'd still like cairo as an option, (behind a flag) but I agree it makes sense to have the easier-to-build pure Haskell libraries as the default.

[bergey]

pandoc-1.13.2 includes the .rst changes.

[bergey]

There are a few related things I want to do when I have some time:

• consolidate imports added by builder, which are currently set 2 or 3 different places
• make diagrams-pandoc recognize a code section of imports for all snippets in the file
• have diagrams-pandoc recognize a code section of code to append to all snippets in the file (see the metafont tutorial for why I want this.)

[cchalmers]

Ah, the changes to the rst reader are much better, although it looks like containers are ignored (would be nice if they where in a div too). It shouldn't be much work to convert the tutorials to .md. I'd prefer if everything was was markdown since it's more popular, simpler and has better pandoc support.

I planned to add other backends to diagrams-pandoc with CPP. It's just -doc I wanted to keep "pure Haskell" backends.

[cchalmers]

Those additions sound nice. I was thinking about using the yaml heading for global options like default backend, extra imports etc.

[bergey]

Adding container directives to pandoc looks very easy. There's also an outstanding issue about internal references: https://github.com/jgm/pandoc/issues/262 which I expect to get to eventually....

[bergey]

pandoc-types is now BSD3, so diagrams-pandoc no longer needs to be GPL. I'd like to make it BSD to match the rest of Diagrams. @cchalmers, if this is merged, you're the only other contributor. What do you think?

No rush. My timeline is Diagrams release 1 April, then sort out new-builder over the next few weeks, then release diagrams-pandoc, if possible.

[cchalmers]

I'm happy to make it BSD3.