Docs with incorrect links

[ncrubin]

When autogenerating the API docs none of the links for "View source on Github" hyperlink are correct. They seem to be missing the / between src and openfermion. For example,

https://github.com/quantumlib/OpenFermion/tree/master/srcopenfermion/circuits/trotter/algorithms/split_operator.py

srcopenfermion should be src/openfermion.

All autogenerated pages seem to have this problem. I assume this is resolvable by changing line 31 in the build_api_docs.py script. Is this correct @lamberta ?

Another pain point is viewing documentation. Documentation seems to be fragmented into the content in the docs/ folder along with whatever is generated by build_api.py. 1) We should make some recommendation to users about the best way to view the markdown. Currently the code says "view on github" which is not helpful if one is developing documentation and wants to check how it will appear. Does Jupyter Notebook or some other program we can run locally view markdown files?

I don't know if it is related to this but some of the links to modules don't seem to work. Also, On the openfermion.md page the "View source" links to the init.py file for OpenFermion. Can this be changed to the readme or something else?

@lamberta let me know where I can read about this documentation infrastructure. I'd like to 1) view all documentation locally on my device 2) understand why links are not working.

Finally, the install.md file references the API docs generator library (https://github.com/tensorflow/docs/tree/master/tools/tensorflow_docs/api_generator) which links directly to the api generator code and not docs. How does one learn how to change the options for this?

[lamberta]

Hi @ncrubin ,

I assume this is resolvable by changing line 31 in the build_api_docs.py

Yes, appears so. Looking at the Cirq build_api_docs.py script and it does include the trailing slash.

Another pain point is viewing documentation

The dev docs are separated into two categories: narrative (guides and tutorials in the docs/ directory) and reference (API ref pages generated from docstrings in the source code). And since docs need to be imported from GitHub and preprocessed a bit to work in the internal system, there's a little bit of a tension with full site previews. Nonetheless, if the Markdown and notebooks render externally, they should also render internally for the site.

See the go/q-docs-cheatsheet for steps to stage internally. There's also a section for how to generate a local copy of the Markdown in OSS used for the API docs reference:

$ cd OpenFermion/  # For site preview, the local git repo directory name must match GitHub repo name.

# Install the tensorflow-docs package for the API generator
$ python3 -m pip install -U --user git+https://github.com/tensorflow/docs

# Generate Markdown from docstrings (use GitHub UI for quick preview)
$ python3 dev_tools/docs/build_api_docs.py --output_dir=docs/api_docs

This Markdown should be previewable in GitHub for a quick check. If you want a complete site stage, follow the additional steps in the above doc.

For notebooks, I think it's easiest to preview in Colab. If it works there, it should be fine for the site. But you can also import a full preview stage if you want to make sure a layout works or something else that's tricky.

Does Jupyter Notebook or some other program we can run locally view markdown files?

Yes, Jupyter provides the nbconvert tool (which we basically wrap and apply our own templates and preprocessing). I saw pandoc also supports notebook conversion now. These could be used for spot-checking notebooks locally but, honestly, I'm not sure that gets you much over just looking at it in Colab. The Open In Colab browser extension is convenient for browsing GitHub.

On the openfermion.md page the "View source" links to the init.py file for OpenFermion. Can this be changed to the readme or something else?

Not sure about that since it's using the docstrings to generate the page. You could add content or hide entries, do you not want this page to show in the ref docs? @yashk2810 or @markdaoust may have suggestions.

let me know where I can read about this documentation infrastructure

The TensorFlow docs contributor section has a few pages:

• https://www.tensorflow.org/community/contribute/docs
• https://www.tensorflow.org/community/contribute/docs_ref

For the internal system, see: go/docs-infra

[lamberta]

Yash confirmed the GitHub source links work. Can you ask on go/tf-docs-chat to clarify? Thanks

[ncrubin]

I think requiring users to push to github to view their documentation changes is a problem. My reasoning behind this is that quite frequently we are writing long mathematical expressions in the docs which don't always translate nicely into rst/markdown. When writing latex a frequent task is to check that the rendered text is formatted as intended. I was able to compile the API docs using the script and everything worked (sans the forward slash bit). I think we need to take the next step and articulate what are good ways to view this markdown. Especially for users without an internet connection. Could you add documentation on how to use nbcovert to view the markdown? I tried opening the markdown in jupyter but it wasn't being rendered properly. The markdown preview on the tensorflow site just allows you to copy and paste markdown. This doesn't really help if you are trying to check links on the full docs or using a local build of the docs to get familiar with the library as a new developer.

I also think we should be explicit about the new documentation strategy. What I mean is that we should probably say "We follow the tensorflow documentation style and build philosophy found here (link1) and here (link2)". ...or something like that. Once I know how to build the FULL docs locally I'm happy to add these things for clarity.

As for the init file for the full package I guess it makes sense it would like to the init file on github.

[lamberta]

To preview LaTeX in Markdown, probably easiest is to open an empty Colab notebook and try it in a new Markdown text cell (Colab supports MathJax): https://colab.research.google.com/notebooks/empty.ipynb If it renders there and not on the site, it's probably a bug in our conversion. But the only way to really know is to do a full preview on the (internal) staging server. You can do that from a local copy without pushing to GitHub, but it's only available to Googlers.

GitHub preview is one way to look at Markdown, but there are other editors or standalone programs that offer Markdown preview. From the docs infra side, we don't really have a preference other than it must render correctly on the website and we do our best to support OSS GitHub workflows. But since we don't own the GitHub previewer, we can't guarantee support (and their notebook preview support is pretty shoddy).

We should loop in @balopat and @dstrain115 for a community docs strategy since they're doing the same to establish best practices for the Cirq. I think it makes sense to use the same for other quantum projects.

Could you add documentation on how to use nbcovert to view the markdown?

We don't use nbconvert directly since it's a part of the pipeline. Our recommendation is for contributors to use Colab to edit notebooks. For the API reference docstrings, we point users here: https://www.tensorflow.org/community/contribute/docs_ref

@balopat Does Cirq have a docs contributor section?

[lamberta]

Ah, OpenFermion already includes a section for building docs: https://github.com/quantumlib/OpenFermion/blob/master/docs/install.md#build-the-docs So, yes, I think it makes sense for the quantum team to agree on a general plan (follow x from the Google dev docs style guide, follow y from the TensorFlow docs guide, and here's the quantum style guide ...). If would be better to have one of these instead of each quantum project maintaining their own.

[ncrubin]

Agreed. @balopat and I can discuss what's best here. It seems like I can use colab with a local runtime (jupyter) so I'll figure that out and write that into the openfermion docs building and viewing instructions. That should be sufficient for viewing the output of the API docs.

[yashk2810]

@balopat was working on a dummy site using some thing. Maybe you can use that.