Generate and upload documentation page on every build

[whereswaldon]

From #26

[ntninja]

Not a technical comment, but wouldn't it be nice to host the docs on IPFS instead (considering what kind of project this is)?

[whereswaldon]

That sounds like a great idea! We can just link to it in the readme over the IPFS.io gateway.

On Aug 15, 2016, at 6:22 PM, Alexander Schlarb notifications@github.com wrote:

Not a technical comment, but wouldn't it be nice to host the docs on IPFS instead (considering what kind of project this is)?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[ntninja]

I'll take care of getting offline docs generation going. Doing it automatically on merge will be another task,

[whereswaldon]

Alright, sounds great. Thanks so much for working on this.

[whereswaldon]

How are you planning to generate the offline docs? We've had a request for pydoc and/or docstring style comments. Is that the route that you plan to take?

On 08/16/2016 12:51 PM, Alexander Schlarb wrote:

I'll take care of getting offline docs generation going. Doing it automatically on merge will be another task,

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/ipfs/py-ipfs-api/issues/58#issuecomment-240164403, or mute the thread https://github.com/notifications/unsubscribe-auth/ADvYzfTBQP2u5ukPzkD0HMjQLJHLR7VBks5qgeqGgaJpZM4Jkyz7.

[ntninja]

Based on your previous comments I think you might be a bit confused about what pydoc actually is (skip the next section if my observation is inacurate), so I'll explain that first:

Unlike most other programming languages python has the ability to retrieve/display documentation for runtime objects (functions/classes/…) while your running the program (so that you don't need a separate docviewer or browser application open to read the help). You can try this on an interactive prompt by doing something like this:

>>> import json
>>> help(json.dump)

which will display the inline help shipped as part of the dump function in the json module. Internally this will read the __doc__ attribute of the json.dump function and then render whatever value it finds (plus some info based off the function's signature) into a new scrollable curses window.

The __doc__ attribute is in turn set by the parser to whatever the value of the first free-standing string in a function, method or class is. Example:

def some_funcy_func():
    """
    Hi I'm some piece of useless documentation!
    """
    …

This would result in the __doc__ attribute of the example some_funcy_func above being set to Hi I'm some piece of useless documentation!; viewable using the help(…). This is also what #5 complained about and which appears to be fixed. :smiley:

What is left to do is to extract these docstrings and render a web page off of the them (since using help(…) is not always appropriate or possible and doesn't give you a nice overview about the API).

–––

I was planning on using the Sphinx documentation suite, that is also used by python itself to generate some nice looking HTML. The only serious drawback about Sphinx is that it insists on RST (which IMHO is a nightmare), but I've figured out how to (mostly) work around that unfortunate fact.

[whereswaldon]

Thanks for the in-depth explanation. I didn't understand the mechanics of pydoc at that level, so I've learned something today.

I think I phrased my question badly. I wanted to know whether you were going to be updating the source code documentation to conform to some doc style other than what it uses now. I had assumed that pydoc was more structured than the current pep257 documentation style. So you're working on getting Sphinx set up and not altering the existing inline docs?

On Aug 17, 2016, at 2:20 PM, Alexander Schlarb notifications@github.com wrote:

Based on your previous comments I think you might be a bit confused about what pydoc actually is (skip the next section if my observation is inacurate), so I'll explain that first:

Unlike most other programming languages python has the ability to retrieve/display documentation for runtime objects (functions/classes/…) while your running the program (so that you don't need a separate docviewer or browser application open to read the help). You can try this on an interactive prompt by doing something like this:

import json help(json.dump) which will display the inline help shipped as part of the dump function in the json module. Internally this will read the doc attribute of the json.dump function and then render whatever value it finds (plus some info based off the function's signature) into a new scrollable curses window.

The doc attribute is in turn set by the parser to whatever the value of the first free-standing string in a function, method or class is. Example:

def some_funcy_func(): """ Hi I'm some piece of useless documentation! """ … This would result in the doc attribute of the example some_funcy_func above being set to Hi I'm some piece of useless documentation!; viewable using the help(…). This is also what #5 complained about and which appears to be fixed. 😃

What is left to do is to extract these docstrings and render a web page off of the them (since using help(…) is not always appropriate or possible and doesn't give you a nice overview about the API).

–––

I was planning on using the Sphinx documentation suite, that is also used by python itself to generate some nice looking HTML. The only serious drawback about Sphinx is that it insists on RST (which IMHO is a nightmare), but I've figured out how to (mostly) work around that unfortunate fact.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[ntninja]

First I'll set up Sphinx. Once that is working, I'll try to convert everything to Sphinx's Napoleon style docstrings (which, unlike RST is actually readable when viewed through pydoc).

[ingokeck]

@alexander255 "The only serious drawback about Sphinx is that it insists on RST (which IMHO is a nightmare), but I've figured out how to (mostly) work around that unfortunate fact."

ReStructuredText (ReST), while older, is very similar to markdown but has the advantage that it is strictly defined (unlike markdown which is a horrible collection of incompatible implementations of a not well defined mess pushed forward by Gruber ) and is extensible (the '..' stuff that probably is the part that confuses people). Rant aside, please have a look at the intro at http://www.sphinx-doc.org/en/stable/rest.html to get a really good introduction to ReST.

[whereswaldon]

@alexander255 Sounds awesome; thank you so much!

My level of care about the documentation style is about 3/10. All I want is something that humans can read that doesn't force you to type redundant information. PEP-257 drove me crazy because it had you summarize each method in a class twice (once at the method level and once at the class level). Anything less verbose than that would be fantastic, whether that's RST or Napoleon.

[ntninja]

@ingokeck

very similar to markdown but has the advantage that it is strictly defined (unlike markdown which is a horrible collection of incompatible implementations of a not well defined mess pushed forward by Gruber )

Hence the CommonMark initiative (and yes Gruber has not been very helpful at all).

I'm aware that ReST isn't that horrible for writing continuous blocks of text (i.e.: the non-API-doc part), but I just prefer MarkDown even for that (using the reCommonMark adaptor library). For inline docs napoleon achieves similar results (and ReST is totally non-readable in this scenario). I believe this combination to be the most readable, flexible and tested approach to obtain nice documentation in both text and HTML format (say what you want about MarkDown but people know how to read and write it, also there is always eval_rst for when you need to call some custom ReST command).

@whereswaldon Sphinx is very smart about redundant information most of the time so this shouldn't be a problem.

[ntninja]

Current progress: fs:/ipns/QmZ86ow1byeyhNRJEatWxGPJKcnQKG7s51MtbHdxxUddTH/Software/Python/ipfsapi/ Any feedback is appreciated!

[ntninja]

I'll keep this open as this issue is more about the actual hosting discussed #63, than about the API docs that were recently added.

[c0llab0rat0r]

Necro post - raising the dead

Since #63 is complete, and we have a script at docs/publish.py to generate and publish documentation to IPFS, all that would remain to automate it would be to identify a target server, secure the target server, and then configure the build (e.g. .travis.yml) with information about how to connect to the target server.

Public values

• Server name
• IPNS public key

Secrets

• Server username (assuming an HTTPS reverse proxy with basic auth)
• Server password
• IPNS private key

Travis has a means to configure secrets on a repo-wide or per-branch basis:

https://docs.travis-ci.com/user/environment-variables/#Encrypting-environment-variables

Updating the IPNS link on every pull request build doesn't make sense (it's intended to match the master branch) but it could publish an IPFS hash on every build. Not sure how useful that is (other than validating that it's possible and works correctly).

If there were a push-button or automated way to trigger a travis-ci run when master is merged, that would be the ideal time to run a process like this.

It looks like travis does support a master-only build step:

https://forum.gitlab.com/t/run-job-in-ci-pipeline-only-on-merge-branch-into-the-master-and-get-merged-branch-name/24195 https://gist.github.com/maxpou/bc3eb944aa727d352f85bc0ce8bc082c https://stackoverflow.com/questions/31882306/how-to-configure-travis-ci-to-build-pull-requests-merges-to-master-w-o-redunda

[ntninja]

Since #63 is complete, and we have a script at docs/publish.py to generate and publish documentation to IPFS, all that would remain to automate it would be to identify a target server, secure the target server, and then configure the build (e.g. .travis.yml) with information about how to connect to the target server.

Weeeeeeell, the docs kinda lie on how the documentation is actually deployed. I have used ipfs-cluster for this for years and I think that could probably we run as part of Travis CI (or any other CI/CD solution).

I also have the server running already (https://ninetailed.ninja/ is an IPFS website).

Setting that stuff up is non-trivial though and I never considered it worth my time, so here we are…