Extracting ipfs mans into their own module, across implenetations

[RichardLitt]

Currently, go-ipfs has the most worked through, thorough, and generally solid documentation for the ipfs CLI app. However, js-ipfs also has a CLI. Rather than manually copying over all of the documentation in a long suite of PRs (as I've done before with, say, the http-api-spec or standard-readme implementation), it may be better to factor out the go-ipfs documentation into it's own module, and then import that into js-ipfs and go-ipfs.

@Kubuxu has started some work on this, and has this json file which basically has all of the commands we'd need from go-ipfs. He suggested that we export docs, opts and args definitions from go-ipfs into YAML, and then rewrite synopsis generator to js and write converters.

What do we think about this? It would mean that the documentation isn't necessarily in the code - or, at least, it'll mean that we have to manually pull and push new versions each time something is fixed. But it should stop extra PRs happening for spelling errors and the like, and should help maintain documentation parity.

[Kubuxu]

I would like to explain why I am proposing to use YAML. As those files will be modified by humans and we need pre-release processing done on them either way (as Go can't read pure data packages) we can use format that will be much nicer to edit.

A little comparison of YAML:

Name: dns
Options:
  - Names: [ recursive, r ]
    Type: bool
    Description: Resolve until the result is not a DNS link.
    Default: true # optional
Arguments:
  - Name: domain-name
    Type: string
    Description: The domain-name name to resolve.
    Required: true # optional defaults to true
    Variadic: false # optional
Help:
  Short: |
    Multihashes are hard to remember, but domain names are usually easy to remember.
    To create memorable aliases for multihashes, DNS TXT records can point to other DNS links, IPFS objects, IPNS keys, etc.
    This command resolves those links to the referenced object.
  Long: |
    Long description here
    4 spaces in the begging will be removed

vs JSON

{
  "Name": "dns", 
  "Options": [
    {
      "Default": true, 
      "Type": "bool", 
      "Names": [
        "recursive", 
        "r"
      ], 
      "Description": "Resolve until the result is not a DNS link."
    }
  ], 
  "Arguments": [
    {
      "Description": "The domain-name name to resolve.", 
      "Required": true, 
      "Type": "string", 
      "Name": "domain-name", 
      "Variadic": false
    }
  ], 
  "Help": {
    "Short": "Multihashes are hard to remember, but domain names are usually easy to remember.\nTo create memorable aliases for multihashes, DNS TXT records can point to other DNS links, IPFS objects, IPNS keys, etc.\nThis command resolves those links to the referenced object.\n", 
    "Long": "Long description here\n4 spaces in the begging will be removed"
  }
}

With the pre-release processing we can also give ourselves a bit more leeway on for example Name vs Names (just use Name and allow it to be array or string).

[daviddias]

Awesome! This will conveniently keep the help docs in sync. I'm personally a fan of JSON, but as Kuba mentioned, YAML can be really nice for editing +:

• since there is always the different between lowercase and uppercase of js vs go properties, having something that pre-processes the thing and makes available both a upper and a lower case version sounds like a good idea to me.
• ability to have several versions of the same doc in different languages, so that then we just have to compile it on the language we want.

Can we kickstart this endeavour by extracting go-ipfs docs into a module, use them in js-ipfs, learn from the experience (try yaml and json) and then feed that into go-ipfs again?

[Kubuxu]

I think it is great idea. I will work on the format but it would be best to just talk it through. Currently I am thinking about YAML file per command with subcommands in directories. Then we can build one uniform JSON from that.

Keys in objects can be small case, it doesn't matter for me.

Any idea for repo name?

[Kubuxu]

I am also thinking about introducing more arguments types (go-ipfs currently uses: files and string) like: number, domain, multipath, multiaddr, multihash. Types could define fallback in case that given type is not handled by implementation (in case of go-ipfs it would be string for them most time).

Another thing: tests and linters, we should keep those documentation files as clean as possible, I know that right now I might be making the project much bigger than it should be but it will benefit us in a future.

[Kubuxu]

Here is (I think) all data from go-ipfs in yaml: https://ipfs.io/ipfs/QmSty2WYzJ84chUVcSL8mLBi91ByKZ5ayo2EPxMWBwZnhq

There are some information missing but it should be solved on case by case basis.

[jbenet]

I'm very much in favor of this approach! 👍

I would encourage you to leave room to experiment for a while before settling on one final syntax.

Also :+1: for YAML. (see also HJSON, the more competent of the "human friendly JSON" things)

[jbenet]

@Kubuxu

• would be fantastic to have those more types in the docs
• those types can enable us to do type checking much more robustly.
• could even code-gen all the argument verification + parsing stuff.
• those types should be the same for the HTTP API spec

[jbenet]

• Also it would help keep the CLIs of both apps the same.

[jbenet]

Oh, i just remembered, take a look at this, it seems pretty fantastic and has impls in the relevant langs:

http://docopt.org/

[Kubuxu]

I don't think that docopt will work for us as for simple number of commands we have.

Other thing is: For options I would like to allow overriding option arguments. To explain: -o FILE - FILE is option argument.

[RichardLitt]

Can we start with this specification for each command, and add more as appropriate?

name: list # Natural language name
command: ipfs ls # Computational name (in path)
options:
  - name: headers # Print headers, normally --headers
    alias: v # Alias, can be also array
    type: bool # Type of argument this option takes
    default: true # Default
    description: Show headers # Natural language description
arguments:
  - name: ipfs-path
    type: canonical-path # accepts also bare hash as /ipfs/HASH
    required: true
    variadic: true
    description: The path to the IPFS object(s) to list links from.
help: # Output of the --help command
  tagline: List links from an object. # Short description that is under 80 chars, generally
  short: null # Description shown with -h
  long: null # Full description

[Kubuxu]

Sometimes we have more aliases, so I would allow alias field to be both a string and array of strings. I would also like to come up with list of types.

We also have arguments and options so we need to account for that.

I will update above example.