Make it easy to publish more detailed documentation to IPFS

[ferrouswheel]

(Splitting out from #104)

There is a field in metadata for a description of the service. This is nice, but it'd be great to make it really easy to publish markdown descriptions. These could then be rendered nicely in a dapp).

Rather than putting this documentation in a json value (urgh!!), we could allow the user to place it as a markdown file inside the services .snet directory (#173). This would automatically be bundled and published to IPFS.

Down the road we might want to allow more complex documentation structures e.g. an entire directory of documentation that can be fetched by service clients and browsed locally. If we want a decentralised developer network, then this is a better option than relying on github or other external websites for decent documentation.

[raamb]

Down the road we might want to allow more complex documentation structures e.g. an entire directory of documentation that can be fetched by service clients and browsed locally. If we want a decentralised developer network, then this is a better option than relying on github or other external websites for decent documentation.

@ferrouswheel I think differently on the support for complex documentation on the platform. I would prefer to have a high level documentation (which the description and URL capture) stored on the platform with the detailed documentation managed by the service provider on their end. We want to store pointers to document stores on our end not the entire document store itself.

[ferrouswheel]

I love rust, the programming language, because it has such a great developer experience. Part of that makes it trivial to fetch and build the docs for any module.

We need to have a better system than a field in a json file (even if it's a single markdown file).

Having each service have a different way of documenting itself will be unpleasant.

If we don't allow it in the metadata, we could make the daemon host the documentation.

At any rate, the definition of the API is part of the value proposition of singularitynet, and without accompanying documentation any API has little value on its own.

It might be something we defer for several release cycles, but I think it's important.

On Mon, 1 Apr 2019, 7:47 PM raamb, notifications@github.com wrote:

Down the road we might want to allow more complex documentation structures e.g. an entire directory of documentation that can be fetched by service clients and browsed locally. If we want a decentralised developer network, then this is a better option than relying on github or other external websites for decent documentation.

@ferrouswheel https://github.com/ferrouswheel I think differently on the support for complex documentation on the platform. I would prefer to have a high level documentation (which the description and URL capture) stored on the platform with the detailed documentation managed by the service provider on their end. We want to store pointers to document stores on our end not the entire document store itself.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/singnet/snet-cli/issues/176#issuecomment-478456458, or mute the thread https://github.com/notifications/unsubscribe-auth/AAHWB-7BcdsmqEimPWnlTn_96GVH5K0Yks5vcauYgaJpZM4aJz0V .

[vsbogd]

One of the options I see is to allow commenting .proto files using Doxygen or other documentation generator and adding separate snet-cli command to generate documentation for the service. I would use documentation generation as documentation which is not bound to actual API tends to being stale.

[raamb]

Here is the approach we are thinking of. During service registration we accept a URL to documentation in markdown format or a document itself. We keep a copy of this on an S3 bucket and serve this on the DApp, the service metadata on IPFS will have a link to our copy and not the whole document itself. The idea behind storing a copy of the doc locally is to ensure that any changes go thru the service curation process as such.

[ferrouswheel]

Still kind of runs counter to what I've been trying to get us to do.

Developers should build and doc their service from their project dir. And push/publish those changes along with everything else. We have too many annoying links and command steps that just are not needed. We should be focussing on building a great product and strip away all unnecessary cognitive load, we're avoiding that and instead creating an ever more complex annoying thing for developers to work with.

But, failing that, we should at least ensure that the doc URL can be an IPFS hash. Otherwise we should bother with ipfs at all.

On Fri, 3 May 2019 at 23:49, raamb notifications@github.com wrote:

Here is the approach we are thinking of. During service registration we accept a URL to documentation in markdown format or a document itself. We keep a copy of this on an S3 bucket and serve this on the DApp, the service metadata on IPFS will have a link to our copy and not the whole document itself. The idea behind storing a copy of the doc locally is to ensure that any changes go thru the service curation process as such.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/singnet/snet-cli/issues/176#issuecomment-489069708, or mute the thread https://github.com/notifications/unsubscribe-auth/AAA5MB7KBGBFDTZ7GOMT5TTPTQRF5ANCNFSM4GRHHUKQ .

[raamb]

I fail to understand your point on

Developers should build and doc their service from their project dir

Depending on the language of the service an appropriate tool can be used to generate documentation from comments and then this documentation can be pushed as part of the service definition. I am fairly certain that I am not on the same page as you on this point so would be great if you can illustrate what you mean. In my mind if I am writing a service using C++ I would use doxygen to generate the documentation and then push that as part of the service registration. Or if I have written a service using Rust would use cargo to generate the documentation and push as as part of the service registration. Are you suggesting that we provide tools to generate documentation for each service?

Now the part about how we store the documentation on the platform. The idea of S3 was it made serving data simple but thats an implementation detail, we can definitely have the file on IPFS and leave it to the client to decide how the content is rendered.

[ferrouswheel]

If we're building a platform of services, I think we should have a documentation publishing as part of publishing a service.

While each programming language has different documentation tools, they don't provide any benefit for documenting the grpc API of a service, which is the level of abstraction SNet provides.

I'm not arguing for spending time building a documentation tool, but making the documentation part of the service publishing/updating is important. If the user has to upload the documentation separately first, and remember to do that every time, people will just fail to keep documentation up to date. I think it'd suffice to just have a local dir or markdown file be defined as the docs, and ensure that's pushed when the service is updated.

[pennachin]

I think both @ferrouswheel and @raamb raise valid points here. @ferrouswheel wants to make it as easy as possible for developers to comply with our documentation guidelines (we need to have those, BTW, for any of these proposals to make sense) while @raamb wants to make it easy for the platform (systems + people) to ensure compliance and the overall quality of services provided.

I think in these early days, we should err on the side of developer convenience, and switch to validation convenience if that turns out to be necessary.

[raamb]

I think it'd suffice to just have a local dir or markdown file be defined as the docs, and ensure that's pushed when the service is updated.

@ferrouswheel thanks this makes it clear to me in terms of your ask. So what we can do is as follows

• Have a folder called docs with a README.md pre-created.
• Service developer can update this
• Service registration will push this folder to ipfs (similar to the proto files)

[ferrouswheel]

On the SDK side, when generating stubs for the services a developer is using, we should also make it easy to pull down the docs of the services used.

Eventually we may want to consider what the naming scheme would look like there e.g. if using a number of services, laying it out like [SERVICE_NAME]/docs/README.md, and having generated proto source in [SERVICE_NAME]/src(for compiled languages that can't dynamically fetch/compile the service_spec)