possibly use mdBook to render md files online

[teonbrooks]

one possibility for the specification now that it is in md is to use mdBook. it's a project from the Rust team and it renders book-like html site.

[choldgraf]

Another option would be to use a simple Jekyll site (e.g. we've been hosting online textbooks with markdown + jekyll and it's worked pretty well as a drop-in for gitbook-style repos). That said, mdbook looks super cool so I'm interested in giving that a shot too.

I think the main question is how much complexity it'd add to this repo. I wonder if it'd be better hosted as a separate repo that pulls from the bids-specification repo?

[chrisgorgo]

Judging from pull request @teonbrooks made (https://github.com/bids-standard/bids-specification/pull/22) it should not add much overhead and will be fairly straightforward.

@franklin-feingold would be great to get your take on this as well.

[chrisgorgo]

One cool feature is that we could include source files such as JSON examples and validate them in CI: https://rust-lang-nursery.github.io/mdBook/format/mdbook.html#including-files

[franklin-feingold]

It may help to split up the specification and the mdbook appears to be a nice appealing way to complete this. I like the way it looks and this can help ease new users into reading it. Including source files is a nice feature. The storage doesn't appear too complex to interpret and follow.

A concern is that splitting it up may make it more difficult to locate items across different sections. We would lose quickly just finding across the entire document. We can also consider keeping the entire specification md file so not to lose the quick finding functionality for those that want the entire specification.

[chrisgorgo]

I had another look at mdbook and there are two features I was missing:

• ability to export a PDF version (a feature requested by @dorahermes)
• ability to provide multiple versions of the rendered specification

I found an interesting potential alternative - https://www.mkdocs.org/.

• It solves the above issues via plugins (https://github.com/jimporter/mike and https://github.com/shauser/mkdocs-pdf-export-plugin).
• It's written in Python (so it would be easier to fix things if we want to)
• It has an export plugin for combining all content into a single page (like @franklin-feingold suggested)
• It has a built-in search function
• Has a beautiful material design theme https://squidfunk.github.io/mkdocs-material/

What do people think?

[teonbrooks]

that could be interesting. maybe we could have a similar test of the intro with mkdocs, compare, choose the better.

[teonbrooks]

* ability to export a PDF version (a feature requested by @dorahermes)

did some searching and there is an export ability using this repo. it adds more to the build, which might be less ideal.

[choldgraf]

Note that I think the choice of build system should be independent of the "split into multiple md files" stuff...so we shouldn't entangle the two if we think it'll be a bit of extra time to find the right build/hosting setup

[chrisgorgo]

Playing with mkdocs right now - looks stunning so far

[choldgraf]

I know that @yuvipanda has used mkdocs before and I think really liked it

[chrisgorgo]

@choldgraf indeed - splitting is a separate issue https://github.com/bids-standard/bids-specification/issues/4

[yuvipanda]

@choldgraf I found mkdocs quite lacking in many ways. Using mkdocs was what finally fully converted me to rST and Sphinx :)

[KirstieJane]

Just on the export as a PDF question - if you click the little print button at the top of the rendered page (that @teonbrooks created in #22 hosted at https://teonbrooks.github.io/bids-specification you can get a pretty nice PDF of the whole site. What are the additional requirements? To do it programatically?

bids-standard-print.pdf

(I just realised that the links are broken in that print out - they're blue but don't link to anything - so that might be a reason to say it isn't good enough)

[dorahermes]

Yes, this little print button is nice! It seems to print the entire spec and not just a sub-section (which would be a problem).

[KirstieJane]

You could just print the page from the browser if you wanted a subsection. What's the usecase that you can imagine for the PDF @dorahermes? To physically print out, or to email around? The broken links are a problem for the email but not for printing for example.

bids-standard-definitions-only.pdf

[dorahermes]

Use cases for having a PDF of the spec: 1) Being able to have a certain version of 'The Spec'. Say you have a project and are putting data in BIDS Version X, I can imagine that a lab would want to have a PDF of this BIDS version alongside instructions of how to put data in BIDS. 2) I personally like having a PDF when I travel a lot and the internet is crappy.

[KirstieJane]

Coolio - I think for the first option it's important to have a nice PDF that includes the links etc. I think this would only have to be made once for each release so maybe it doesn't have to be done really automatically.

For crappy internet the pdfs with the broken links would probably be fine.

Totally ok by me for the search for a nice way of doing everything to continue, I guess this is just a light +1 for the simplicity of mdBook 😸

[dorahermes]

and one more example,

3. Some people might need to be able to generate a paper-trail for data-management plans.

+1 for a nice PDF with links only generated once per release

[chrisgorgo]

Good find @KirstieJane.

@yuvipanda what were the ways mkdocs was lacking in?

BTW the search feature in mkdocs is pretty neat

[teonbrooks]

@satra brought up another great suggestion for a rendering engine, sphinx: https://github.com/bids-standard/bids-specification/pull/28#issuecomment-426812108

it seems that there are three options at the moment

• mdBook
• mkDocs
• sphinx

[chrisgorgo]

I created some prototypes:

• mdbook version https://github.com/bids-standard/bids-specification/pull/37
• mkdocs version https://github.com/bids-standard/bids-specification/pull/36

Let me know what do you think

[nicholst]

I'd like to throw one consideration into the formatting discussions: HTML print-to-PDF by the browser is a highly browser-dependent operation. E.g. I've even seen different PDFs produced from the same website on the same OS but slightly different versions of a Chrome browser.

My $0.02 is that a highly professionally rendered PDF is a priority for the spec. If the plan is that each user/reader is to create a PDF by a browser Print-to-PDF action, we need to verify that good looking PDF is found with different browsers (presumably at least Firefox, Chrome, DefaultOSbrowser).

Thus, if one of the tools we're evaluating has the capability of rendering to PDF directly (in a browser-independent fashion), this is a big plus.

See, e.g. first page on Mac OS on, Chrome, Safari, Firefox, illustrated here, respectively:

Full PDFs here: Chrome | Safari | Firefox

[satra]

@chrisfilo - i prefer the simplicity of the first one #37

@nicholst - i'm personally fine with just reading html, but i agree that if a pdf is to be produced a consistent version would be good.

[chrisgorgo]

@nicholst delivering professionally looking PDF will be hard. Best approach is using pandoc, (see example output https://11-151034407-gh.circle-artifacts.com/0/home/circleci/project/site/specification.pdf) but there still issues with tables and header hierarchy.

I was hoping that most people will be interested in HTML outputs.

[choldgraf]

2 cents about the PDF printing: I think this should be community-driven question. How do we know that a large % of users want to print the specification out as a PDF for reference? Perhaps a user survey of some kind can help give us a path forward.

[nicholst]

I may have muddled things. IMHO, a polished PDF adds credibility/stature to the effort, but if the consensus is that HTML is sufficient, that’s fine. I think my point is that if we’re considering PDF it should be good & consistent; otw, we say this is a HTML spec and leave it at that.

As a random idea: I just looked at some PDF from Read The Docs and it looked very nice. Is that system in the running here?

[choldgraf]

we had discussed using readthedocs and hosting with sphinx. I believe it actually works with mkdocs https://mkdocs.readthedocs.io/en/stable/ so maybe that's a good approach!

[chrisgorgo]

Would someone be willing to explore the readthedocs route (especially the PDF generation feature)?

[choldgraf]

if you give me admin privileges on the repo and merge the mkdocs PR (so that the mkdocs files are in docs/, then I can try to set up RTD.

[chrisgorgo]

Would you mind doing this on your fork? I don't want to commit to mkdocs solution yet.

[nicholst]

@choldgraf: Oddly that link https://mkdocs.readthedocs.io/en/stable/ lacks the PDF rendering I had seen.

I was looking at this site: https://packaging.python.org/ which has a floating pop-up menu in the lower right that gives a PDF download option. In contrast, say, https://mriqc.readthedocs.io/en/stable/index.html has a similar pop-up (on the lower left) but offers no PDF download.

[choldgraf]

hmmm - usually it shows up in the little "latest" menu in the bottom right (e.g. it's there on this page https://repo2docker.readthedocs.io/en/latest). Maybe RTD doesn't build PDF with mkdocs?

[choldgraf]

@chrisfilo isn't there already a PR open that gets this repo built on mkdocs? All we'd need to do is set up readthedocs with whatever repo that is. What is the question you'd like answered with respect to readthedocs? Just whether you can download as PDF using mkdocs?

[chrisgorgo]

Correct - there is PR that adds mkdocs build, but more needs to be done to make it work with RTD. Just hooking it up with RTD leads to this result: https://bids-specification-demo.readthedocs.io/en/latest/. Even though there is no index and PDF is basically empty the individual pages got rendered (see https://bids-specification-demo.readthedocs.io/en/latest/src/04-modality-specific-files/01-magnetic-resonance-imaging-data.html). However, tables are not displayed correctly (which might be an intrinsic limitation of RTD support for markdown.

So a host of problems - what needs to be investigated is if they can be mitigated easily leading to quality HTML and PDF output.

[choldgraf]

ok - if I have a moment than I'll give it a shot

[chrisgorgo]

Thanks!

On Mon, Oct 8, 2018 at 11:37 AM Chris Holdgraf notifications@github.com wrote:

ok - if I have a moment than I'll give it a shot

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/bids-standard/bids-specification/issues/18#issuecomment-427881434, or mute the thread https://github.com/notifications/unsubscribe-auth/AAOkpxX-Zi5Z3yZLcqNT7W6FuYj2xB18ks5ui3ExgaJpZM4XDOlu .

[choldgraf]

So an update here: it looks like mkdocs is pretty easy to set up on readthedocs, but it does not support anything like PDF/EPUB generation of the document.

What do folks think about this?

[teonbrooks]

@choldgraf, do you know how repo2docker generates its pdf and epub? that would be a nice to have if it's easily implementable.

[choldgraf]

We're using sphinx there, which uses rst under the hood. Sphinx works with markdown as well through Recommonmark, but I think some features (like markdown tables) don't work :-/

[chrisgorgo]

Even though I like the option of having PDF (if it comes 'for free'), but it seems this is not easy. I believe we get the "prestige" effect from multiple papers related to BIDS that are published and in the works. I'm not sure we need to do a formal survey, but happy to be convinced otherwise.

Personally, I would just pick one of the HTML only solutions, for now, deploy it, but be open for future contributions enabling quality PDF output.

[teonbrooks]

+1. This would be in the same vein as W3C.

for archival purposes, I think as long as we can have pointers to the previous versions, we should be set. we can just have links to all the pdf versions before markdown, and then we will just have versioned html build views of post markdown.

[dorahermes]

+1 on being pragmatic and +1 on Teon's suggestion on having a previous version archived

[choldgraf]

A note re: readthedocs, it will also archive old versions of the specification relatively seamlessly, so if we want to use mkdocs (#42) then that would come "for free" even if PDF generation does not

[chrisgorgo]

Does this work with mkdocs integration? I could not see the menu with versions usually present in sphinx/rst readthedocs documents.

It's also not so hard to configure circle to deploy each version to a new subfolder (1.1.1/index.html, 1.2.0/index.html). Archival versions of source documents (Markdown) will, of course, be available on github as well.

[nicholst]

+1 for accepting this as a HTML-based standard.

[choldgraf]

I believe that versioning should work fine with mkdocs...if the community is +1 on using mkdocs then I can try to confirm this in my RTD PR

[chrisgorgo]

If versioning via RTD doesn't work we can always use https://pypi.org/project/mike/ and push directly to gh-pages.

I don't have a strong preference - I like material design of mkdocs, but I dislike the necessity of having an extra index.md file and lack of single page print option.

[choldgraf]

@chrisfilo this makes think of a question: does the bids-specification community have an official governance process? E.g., what's the process by which we would decide to use one build system over another?

[chrisgorgo]

We listen to each other, strive for consensus, use voting to resolve conflicts.

However, there is no formal governance structure. It's a complex topic that we will address in due time. If you have concrete ideas for a governance structure I would love to hear them (for the sake of clarity in a separate issue).

On Wed, Oct 10, 2018, 5:33 PM Chris Holdgraf notifications@github.com wrote:

@chrisfilo https://github.com/chrisfilo this makes think of a question: does the bids-specification community have an official governance process? E.g., what's the process by which we would decide to use one build system over another?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/bids-standard/bids-specification/issues/18#issuecomment-428740267, or mute the thread https://github.com/notifications/unsubscribe-auth/AAOkp68XR2iV0euzQiVCxhyXmZBD3QNGks5ujmekgaJpZM4XDOlu .

[choldgraf]

Sounds good - longer governance conversations I agree are a separate issue.

For this issue, my 2 cents is that we should:

• just use mkdocs for now
• use readthedocs for hosting and versioning
  • If RTD won't work for versioning, auto-deploy it to folders per @chrisfilo's suggestion
• keep considering another markdown-based SSG like mdbook for a future decision
• choose a URL that is not RTD-specific (maybe?)

My reasoning: The mkdocs PR is already ready to go more or less, and I don't think we should let "picking the right tool" get in the way of "picking any tool at all". With mkdocs we aren't committing ourselves to any kind of complex dependency that'll make it hard to switch to something later if a better option surfaces (beyond potentially the reliance on RTD for versioning...on that point, we should probably get a URL that isn't dependent on readthedocs so that we could switch if need be). I am still intrigued by @teonbrooks 's idea of mdbook, but I think we could keep exploring that idea on an ongoing basis and revisit this conversation later if it seems like it's the best choice.

Thoughts?