Generate PDF spec

[dignifiedquire]

edit: PDF is no longer supported, see https://github.com/filecoin-project/specs/issues/266#issuecomment-579067300.

---

Probably using pandoc, based on the generated html is the easiest way.

[pooja]

Some additional notes for whomever picks this up:

• Every time spec master changes, we want a new PDF output of the spec to be generated and placed in a visible place in the repo (maybe releases?)
• If we can get the README to update with every update to master and link to the latest release, that would be awesome

[mslipper]

Ok, I took a look at this and here's where I'm at:

• The existing PDF generation tool through gitbook depends on a binary that's only available as part of a Mac app bundle called Calibre, so we can't automatically generate PDFs using this tool unless the build system is running on a Mac as well.
• I was able to use pandoc as suggested to generate a PDF. I wasn't able to get it working with the generated HTML, however, but was successful generating it from the included Markdown files. I've attached the generated PDF to this issue for your approval: spec.pdf.

There are a couple places where formatting breaks in the generated PDF, specifically:

1. Areas where long hex strings are represented in tables (the Latex engine doesn't know where to wrap them, so it doesn't).
2. Code blocks that aren't preceded by a line break.
3. Documents that contain invisible UTF-8 characters like \u200b.
4. Deeply nested listed such as those in drgporep-circuit.md

I can fix all of the above manually with the exception of number 1; my Latex isn't good enough to properly format those strings and those strings alone without breaking other formatting.

Finally, I wrapped the generation command in a Bash script that'll insert a version number based on a CLI argument. The tough thing about committing this to master on every update is that this will itself cause an update to master. Instead, what do you think about running the generation script when we tag a specific spec version? The tag will up in the 'Releases' link on GitHub, and include links to the generated PDFs as requested above.

Let me know how you'd like me to proceed; I can have a PR out late tomorrow with the script and fixes to the documents as necessary if you're a fan of the PDF's formatting and my proposed solution regarding tagging.

[dignifiedquire]

This is looking good :)

We should fix

• code blocks not preceded by a line break,
• invisible utf-8 chars
• and deeply nested lists

directly in the source of the markdown files.

In regards to the hex strings I am not sure, I think they are in tables atm, maybe sth like this could work: https://tex.stackexchange.com/questions/318872/automatic-line-breaks-in-a-table ? not sure how to pull this into the process

[pooja]

Thanks a bunch @mslipper! Sorry for the delay on this.

Can you clarify the flow here? When we tag a specific spec release in GitHub releases, that will kick off a PDF generation process? Where will those PDFs be found -- also in the Releases?

@dignifiedquire Can you confirm whether you are currently working on https://github.com/filecoin-project/specs/issues/264? If not, @mslipper can we find a way to merge these two issues, i.e. maybe when we have a new release for the spec, we can both re-deploy the website and generate corresponding PDF output?

If this is the only way to get this to work, that is fine, but I would really prefer for us to be able to generate the PDF output and auto-deploy the website on any change to master. Could we host the PDF output on the spec website or something as a way to workaround this? I know it's not ideal, but that'll make it a lot easier than having to keep pushing new spec releases, which is not part of our workflow today.

[dignifiedquire]

I am not working on it atm

[mslipper]

Sure. The proposed flow is to use the GitHub 'Releases' screen to host the built PDFs, so the full flow would be:

1. Generate the PDFs from master.
2. Tag a release (i.e., via git tag -v<version>)
3. Push the tag.
4. Draft a new release from the 'Releases' screen, and upload the generated PDFs as assets so that they show up as binaries. It'll look similar to this:

That said, if the goal is to upload the PDFs somewhere else on every update to master (but without committing them into the repository), I can make that happen relatively easily. I'll add that to this PR. The only reason for the somewhat convoluted flow above is to commit the PDFs into the repo.

[mslipper]

@pooja I've uploaded a new version of the PR that will deploy a new version of the PDF alongside every new version of the site. It puts the link in the 'Overview' section on the first page.

[pooja]

Looks great!

Two requests:

• Can we change the text to "You can also download the full spec in PDF format"?
• Can we move "You can also download the full spec in PDF format" to right below the warning?

Could you deploy both the GitHub tagging method and this auto-deploy method? Interested in the feasibility of this. If it's impossibly hard, then just the auto-deploy method works well!

[pooja]

Also, what will the PDF title be? Assuming it's autogenerated, what are the parameters?

[mslipper]

Will make both the changes above shortly. The title is "Filecoin Specification \<commit hash or version>".

[mslipper]

Changes done.

[pooja]

@mslipper I don't see the ability to download the PDF in the specs website. Do we need to do anything else to complete this?

[dignifiedquire]

That's likely because the site needs a redeploy, let me do one quickly

[dignifiedquire]

make publish worked, but the link is empty

[dignifiedquire]

 make publish
git submodule update --init --recursive
./publish.sh
Deleting old publication
Checking out gh-pages branch into public
Preparing worktree (resetting branch 'gh-pages'; was at 7e938db)
Branch 'gh-pages' set up to track remote branch 'gh-pages' from 'origin'.
HEAD is now at 7e938db Publishing to gh-pages (publish.sh)
Removing existing files
Generating PDF
[WARNING] Could not parse YAML metadata at line 1214 column 1: :8:0: Unexpected '`'
Error producing PDF.
! LaTeX Error: File `adjustbox.sty' not found.

Type X to quit or <RETURN> to proceed,
or enter new name. (Default extension: sty)

Enter file name:
! Emergency stop.
<read *>

l.98 \usepackage

cp: pdf-build/ is a directory (not copied).
Generating site

                   | EN
+------------------+----+
  Pages            | 28
  Paginator pages  |  0
  Non-page files   |  0
  Static files     |  9
  Processed images |  0
  Aliases          |  0
  Sitemaps         |  1
  Cleaned          |  0

Total in 361 ms

[pooja]

@mslipper Can you take a look?

[jbenet]

i believe this was fixed after these comments, but then broke again?

[schomatis]

@mishmosh Is PDF supported? I'm not seeing any references to it neither in the README nor in the website, or was this issue close because we're not supporting PDF generation/download? (The motiviation for this question is that I'm having trouble loading the website, https://github.com/filecoin-project/specs/issues/860).

[mishmosh]

@schomatis correct, PDF is no longer supported but we're committing to a usable & searchable website. Thanks for the #860 report. I replied over there and we will investigate.

[citizen-stig]

This is very sad that generating of PDF is not supported. eInk readers excluded as a class.

Is there a way to bring it back, at least not in CI, but manually generate it from source?