Build system 0.0.1

[michealbenedict]

Addresses Issue #8

[Dependencies: Pandoc and TeXLive|MacTex] Quick heads up: I used gulp instead of grunt to play with it. I am willing to change back to grunt if required.

This PR adds a primitive build system to generate the book in pdf and html formats. Thanks to the use pandoc, this book can be exported to various formats with the addition of simple gulp tasks. I have also listed the requirements, constraints I assumed for my proposed solution. Refer to README.md for instructions on how to use the gulp tasks to build the book. Happy to iterate based on feedback

Requirements

1. Ability to add and set order for topics and chapters easily.
2. Ability to export the book to various file types.

   Constraints:

3. Ability to add a new adhoc directory structure for new chapters or topics. The build system must cater to these as well.

   Proposed solution:

4. Centralized file (chapters/toc.md) which lists the order of how the content is structured in the book (granularity: individual topics)

This could have been a simple JSON file as well since the main purpose is to define how the content should be exported (as pdf,html or any other file format), but I chose markdown to be consistent with the content. I also felt having a toc.md would be flexibile to generate a (well linked) toc portion of the book easily.

I used pandoc since I felt that it addressed the need to export to various file types as required.

Pros

• Easy to
• Easy to add/remove chapters (and/or topics)

  Cons

• Can grow to be large (and could potentially be hard to maintain)
• May limit flexibility in terms of how the book can be generated. For example, How can I generate a rich HTML site with custom design?

[dashed]

On this point:

May limit flexibility in terms of how the book can be generated. For example, How can I generate a rich HTML site with custom design?

Pandoc has a css and template flag.

[dashed]

IMO, I think toc.md should be a JSON file, which makes preprocessing so much easier by slimming up gulpfile.js (i.e. no need for all that parsing).

If you need to, you can easily generate the markdown from the JSON as needed.

[sindresorhus]

May limit flexibility in terms of how the book can be generated. For example, How can I generate a rich HTML site with custom design?

gulp-markdown-pdf supports custom CSS: https://github.com/alanshaw/markdown-pdf#optionscsspath

IMO, I think toc.md should be a JSON file, which makes preprocessing so much easier by slimming up gulpfile.js (i.e. no need for all that parsing).

https://github.com/tooling/book-of-modern-frontend-tooling/pull/31#discussion_r9368917

[michealbenedict]

Summarizing the discussion around pandoc and non-pandoc solution (npm modules) to help come to a consensus.

AFAIK, the formats we are looking to export this book are

1. PDF
2. HTML
3. EPUB
4. MOBI

Pandoc

@sindresorhus is concerned regarding speed and ease of installation My qualm with pandoc is the requirement of TeXLive/MacTex to help generate PDF. Apart from that, pandoc is a very well supported tool which works in a predictable manner across all three platforms (win, osx and linux).

Non-Pandoc solution

I am particularly concerned regarding the support when opting to use npm modules. HTML (node-markdown) - project has a large community and is supported well PDF (markdown-pdf) - seems to be updated frequently I was unable to find well established projects to help export to EPUB, MOBI formats, any recommendations?

I am leaning a bit towards pandoc.

[michealbenedict]

IMO, I think toc.md should be a JSON file, which makes preprocessing so much easier by slimming > up gulpfile.js (i.e. no need for all that parsing).

If you need to, you can easily generate the markdown from the JSON as needed.

@Dashed Do you have any thoughts around moving to semantic file naming as suggested in #21 and based off @sindresorhus https://github.com/tooling/book-of-modern-frontend-tooling/pull/31#discussion_r9368917? I'll soon be submitting a patch for this.

[sindresorhus]

@rowoot i would prefer both. Pandoc for epub/mobi and gulp-markdown-pdf/gulp-markdown for pdf/html. That way we can skip the tex dependency, while still being able to produce all wanted formats.

[dashed]

@rowoot I think semantic file naming may be the way to go -- though I don't think it may matter when you consider something like the following for JSON:

[
  {
    "name": "Introduction",
    "file": "path/to/file"
  },
  {
    "name": "Build Systems",
    "toc": [
      {
        "name": "Modern tools vs shell scripts",
        "file": "build-systems/modern-tools-vs-shell-scripts.md"
      }
    ]
  }
]

If it's possible to have metadata in the markdown files, then you can just outsource the name property to the respective file. Then with this and the semantic file naming, you wouldn't need the toc.json anymore since you can just recursively traverse the directory tree, and inspect the metadata of each file.

So, something like https://gist.github.com/Dashed/8792825 would allow custom frontmatter in YAML for markdown files. All it does is extract frontmatter. You'll need another transform stream to parse them when building the toc.

[michealbenedict]

I tried both the approaches, and here is my feedback.

• Semantic naming based approach
  • Naming file with numbers makes sorting simple (programmatically), but introduces pain for a user when adding a new topic or a chapter. This can lead to commits with bulk diffs :/
  • This approach would require a custom frontmatter (as @dashed suggested) to help store meta-data associated to chapter/topic. This is somewhat good (despite the need for a custom frontmatter) since files can be adhoc and do not require an "entry" in something like toc.json.
• TOC based approach (JSON)
  • File specific meta data can live in this one file (but is an extra entry by the user)
  • This can grow fairly large and can become hard to navigate compared to the current markdown based toc. I personally prefer the markdown based TOC approach which looks much cleaner albeit the added overhead of preprocessing in gulp.

After re-evaluating, I am leaning towards the TOC based approach because of its simplicity (from the perspective of a user who is contributing to the book and the folks who would review the PR). Thoughts?

[dashed]

Just curious, what does your JSON object look like for the TOC based approach?

[michealbenedict]

toc.json = https://gist.github.com/rowoot/8798003 toc.md = https://github.com/rowoot/book-of-modern-frontend-tooling/blob/feautre-gulp-pandoc/chapters/toc.md

[dashed]

Looks like the JSON file can get gnarly really fast. What if you store it in YAML format? Then convert to JSON when importing it. I think either JSON or YAML would still be verbose.

Looking at the alternative solutions, it looks like, in my opinion, that the markdown format (as a list) looks to be the way to go -- in terms of readability and maintainability. At this point, it would be consideration between of having custom markdown parser and the verbosity that the JSON approach requires.

I feel that having a table-of-contents metadata file should somehow be unnecessary -- but it wouldn't be a big deal if it's easy to maintain.

[michealbenedict]

What if you store it in YAML format?

Readability would still not match the markdown approach.

At this point, it would be consideration between of having custom markdown parser and the verbosity that the JSON approach requires.

My focus is to make the process of contributing content to the book super simple. Thus I am leaning towards using markdown (over JSON) for the TOC approach even though there is some minor preprocessing (parse the markdown file) using gulp.

I feel that having a table-of-contents metadata file should somehow be unnecessary -- but it wouldn't be a big deal if it's easy to maintain.

Along with being a metadata file, It also helps in structuring the chapters/topics when exporting to different formats (no need for semantic naming).

[dashed]

My vote's on the markdown approach of TOC as is.

[michealbenedict]

Changelog:

• Added EPUB gulptask which uses pandoc underneath (/cc @addyosmani)
• Moved the markdown based TOC parser (decided to stick with this, refer to above discussion) to a separate gulp-plugin directory
• Added build-assets folder (which contains the pdf.css to help generate pdf files)
• Moved to using gulp-markdown and gulp-markdown-pdf instead of pandoc for generating HTML and PDF (/cc @sindresorhus)

Usage:

$ gulp
[gulp] Running 'default'...
[gulp] Available Tasks:
[gulp]   generate:html
[gulp]   generate:pdf
[gulp]   generate:epub
[gulp] Finished 'default' in 1.5 ms

[michealbenedict]

(ping) .. does this look good to merge?

[dashed]

I noticed there is no task for individual HTML pages. I only see a concatenated, conglomerate version.

[michealbenedict]

I did think about it, but descoped it at the moment till I can find out better mechanisms (along with pros and cons) of exporting it as a site.

For now, removing the concat() gulp task can export it as individual HTML pages.

[dashed]

Does the markdown-to-html plugin support templates?

[addyosmani]

This looks good to merge but will need a little rebase work. Once we're all green on this side I'd be happy for us to land this.

[michealbenedict]

@Dashed What do you mean by templates? The gulp markdown task uses https://github.com/chjj/marked library if this helps.

Changelog:

• Moved to using through2 over through (/cc @sindresorhus)
• Merge with master (includes latest docs for brunch, grunt, gulp). Also removed some unwanted files in the process. Refer to commit 4634836

A draft PDF version of the book (currently not styled), https://dl.dropboxusercontent.com/u/32194349/book.pdf

[addyosmani]

Thanks for all your work on this @rowoot. Landed!

[michealbenedict]

Thanks @addyosmani @sindresorhus and @Dashed for all the feedback! Just FYI, this PR addresses #21 and #8 (feel free to close the issues or let me know if there is something missing).