Enable all trusted committers to produce an ebook of proven patterns

[spier]

Our mission statement says:

Continuously publish the most mature patterns as an ebook

@dicortazar has produced an updated gitbook in April 2020. This is an interactive (clickable) version of our patterns: https://dicortazar.gitbook.io/inner-source-patterns/ (has been removed in the meantime)

Possible Challenges (with the current approach)

• people cannot download the gitbook. When I heard "ebook" I was expecting an ebook/PDF that people can download. Maybe that expectation is not shared in this group?
• right now only @dicortazar can update the gitbook. That will lead to the gitbook being outdated quickly. I think it would be great if all trusted committers could update the ebook. Even better if that would happen automatically when changes are merged to this repo.
• the gitbook does not seem to be an official output artifact of the InnerSource Patterns community yet. For example it is not shared on innersourcecommons.org/resources/books/ yet, which makes me think that it is sort of a beta release so far?

Proposed Solution

• produce a PDF and publish that at innersourcecommons.org/resources/books/
• automate the production of the ebook, optimally so that it automatically refreshes when the master branch of this repo is updated
• unclear: What to do with gitbook? Do we want to have both (gitbook and a PDF) or only one?

Tech

@MaineC shared how to generate a simple PDF using pandoc. The pattern .md files are listed explicitly to guarantee a certain order.

pandoc start-as-experiment.md dedicated-community-leader.md praise-participants.md project-roles/trusted-committer.md service-vs-library.md 30-day-warranty.md contracted-contributor.md crossteam-project-valuation.md common-requirements.md gig-marketplace.md discover-your-innersource.md improve-findability.md innersource-portal.md introducing-metrics-in-innersource.md junkyard-styled-innersourcing.md modular-code.md review-committee.md -o patterns.pdf

Looks like this could be further automated e.g. using pandoc-action-example

References

• more details in this Slack Conversation

[spier]

@dicortazar I would be really curious to hear your opinion about the challenges and proposal above, as you have all the working knowledge from producing the gitbook so far.

[spier]

Some other relevant links if we want to use pandoc for this:

• Writing a Book with Pandoc, Make, and Vim (and hackernews discussion)
• Using Pandoc to publish a book in multiple file formats: experiences and wishes
• Customizing pandoc to generate beautiful pdfs from markdown

[lenucksi]

Thanks for the write-up @spier! I did a bit of searching here too and found this nice LaTeX template to be used with PanDoc:

• https://github.com/Wandmalfarbe/pandoc-latex-template
• Pandoc themselves keep this list of contributed templates: https://github.com/jgm/pandoc/wiki/User-contributed-templates
• General Pandoc demo: https://pandoc.org/demos.html

Here's something more, German language though

• https://vijual.de/2019/03/11/artikel-mit-markdown-und-pandoc-schreiben/
• https://baireuther.de/page/pandoc/

And matching Docker images (or their source repos) we can hopefully build upon:

• https://github.com/pandoc/dockerfiles#available-images
• https://gitlab.com/raabf/dockertex/

One commercial service rendering books from markup: https://bookdown.org/

[spier]

So many options :) Great to know that pandoc can also do epub.

I will try to find something that can use a GitHub Action to build both a PDF and an epub. Let's see :)

[lenucksi]

So many options :) Great to know that pandoc can also do epub.

I will try to find something that can use a GitHub Action to build both a PDF and an epub. Let's see :)

If there isn't any GitHub action will actually happily run any Docker container or even blank shell scripts. So if there isn't anything readymade in the marketplace or directly usable from within the Pandoc Docker repos we can just hack our own...

[spier]

Another example of a gitbook produced from a github repo:

• repo: https://github.com/OWASP/owasp-masvs
• book: https://mobile-security.gitbook.io/masvs/

[spier]

Btw I am starting to look into gitbooks today. If there is any gitbooks expert that sees this here and wants to help, let me know :)

[lenucksi]

Btw I am starting to look into gitbooks today. If there is any gitbooks expert that sees this here and wants to help, let me know :)

Sounds great, @dicortazar already has a bit of GitBook experience. We also have a non-profit commercial subscription for it, that will likely come in handy.

Looking forward to see a WIP PR for it so others can join in. Could you try to push that to a different branch than master for now so we can possibly merge more material while iterating on the rendering?

Found this GitHub Action that might be useful for it: https://github.com/marketplace/actions/gitbook-action

I also took a look at the OWASP example, that's a great find! 👍

• They render fully automatic using GitHub Actions
• Their fixed artifact (PDF / EPUB / ...) rendering is based on Pandoc, ConTeXt and the Eisvogel template all done & integrated. That's something awesome we hopefully can recycle.

[spier]

Yep, I am already working on a branch book (have not created a PR yet).

For now I am shooting for a gitbook only (that will also provide a PDF download).

It looks like I can create that directly with the GitHub integration available in gitbook, but maybe that GitHub Actions offers more flexibility. We can check that.

I will share what I have once I have learned a bit more about it. Latest next week.

[spier]

In an effort to release early and often, here a first experiment: https://innersourcecommons.gitbook.io/innersource-patterns-staging/v/book/

It is based of this branch: https://github.com/spier/InnerSourcePatterns/tree/book

I will send a PR later and summarize what I have tried, and how this book works right now.

[lenucksi]

Looks good to me. I think I'll give the Pandoc magic of OWASP a go once I find a bit of time. I should give us a bit more flexibility when it comes to nice, book-style rendering. Not sure what Gitbook offers us in that regard.

Looking forward to the PR. Feel free to also just open up a WIP-PR if you like.

[spier]

Indeed it looks like gitbook offers very few layouting options. Since they moved their service away from their CLI approach and exclusively to a SaaS offering on gitbook.com they have greatly reduce the amount of configuration options for their service.

I am still a bit unsure if the pandoc approach will be sustainable for us, assuming that it will be more complex to setup and maintain (and we have very few maintainers right now :)).

One option could be to do run with the gitbook approach for a little while, which helps us to create the missing guidelines for level 2 and 3. If we then realize that the content is good but the format isn't pretty enough, we could still look at the pandoc approach in addition to gitbook (or maybe even as a replacement).

Will push a WIP PR now.

[lenucksi]

Indeed it looks like gitbook offers very few layouting options. Since they moved their service away from their CLI approach and exclusively to a SaaS offering on gitbook.com they have greatly reduce the amount of configuration options for their service.

Great thanks for the insight! Can you point to some point of their docs detailing the options/restrictions around PDF layout they provide?

I am still a bit unsure if the pandoc approach will be sustainable for us, assuming that it will be more complex to setup and maintain (and we have very few maintainers right now :)).

It's not more complex than what we already use to render the learning path, effectively supported by a large OSS organization and highly popular, proven software. And who knows it might even attract more people that are interested in this - especially given that this will enable us to render both Patterns and LP content to pretty PDF and EPUB files.

One option could be to do run with the gitbook approach for a little while, which helps us to create the missing guidelines for level 2 and 3. If we then realize that the content is good but the format isn't pretty enough, we could still look at the pandoc approach in addition to gitbook (or maybe even as a replacement).

Both approaches are entirely independent. Gitbook is fine for pretty online rendering and apparently some PDF rendering until we have something better.

If this double approach works for OWASP, it works for us, especially since we can pretty much copy their approach.

Will push a WIP PR now.

Great, thanks.

[spier]

Indeed it looks like gitbook offers very few layouting options. Since they moved their service away from their CLI approach and exclusively to a SaaS offering on gitbook.com they have greatly reduce the amount of configuration options for their service.

Great thanks for the insight! Can you point to some point of their docs detailing the options/restrictions around PDF layout they provide?

So the formatting options for text and images can be found here: https://docs.gitbook.com/editing-content/rich-content

It's pretty basic stuff that is mostly also available when formatting content in markdown.

I have not seen any specific formatting options for the PDF output actually. gitbook just offers an "Export as PDF" option and that seems to be it.

[spier]

While producing a first version of the gitbook in https://github.com/InnerSourceCommons/InnerSourcePatterns/pull/217, I was sometimes second-guessing whether this activity makes sense, if we should just keep the Patterns as a GitHub repo and not do a gitbook at all.

Eventually I decided that producing a gitbook is still worthwhile, for the following reasons:

• (consumers) have something that is “nicer” to read than things on GitHub (we could try to qualify what makes gitbook "nicer")
• (consumers) have stable URLs for patterns i.e. even if the pattern files are moved in the folder structure in the repo, the URL of the pattern can still stay the same. We even have the option to add in redirects, should we want to rename a pattern.
• (consumers) export book as PDF and read on other devices, or maybe even print
• (producers) a motivation for pattern authors (and all contributors) to level up patterns from 1-initial to higher level patterns. Only 2-Structured and 3-Validated are published in the book.
• (producers) a motivation to introduce more specific quality guidelines for level 2+3, as that will help all contributors to level up patterns faster. This should result in even higher quality, and make us all feel even more proud of the content in the book.
• (producers) learn which patterns get viewed most frequently (e.g. through Google Analytics)

[spier]

The first gitbook generated from the master branch of this repo is now live at: https://innersourcecommons.gitbook.io/innersource-patterns/

We may still want to configure a custom domain for this book and other things of that nature. Those tasks and ideas are captured in https://github.com/InnerSourceCommons/InnerSourcePatterns/pull/242.

I think we can already close this issue though.

[spier]

We do have an ebook/website now at https://innersourcecommons.gitbook.io/innersource-patterns/, so I am closing this issue. Any follow-up work can go into new issues.