apidocs

[oxarbitrage]

http://apidocjs.com/

MIT licensed documentation page generator similar to doxygen but with focus on documenting the api calls. It is currently being implemented by eos between other exchanges with api access. it will provide new developers a familiar and clean api website.

there are other options around like: https://github.com/lord/slate

apidocs is easy to use (example generated very quick: http://open-explorer.io/apidoc/) but we need to add specially formatted comments of the calls into the cpp files. as our database_api.cpp and api.cpp are already big i also propose to refactor and split in different files by groups just as account_api.cpp, objects_api.cpp and so on. this will make some progress in: https://github.com/bitshares/bitshares-core/issues/95

i didn't checked how hard it will be to separate the apis into files without breaking anything or what will be the best approach.

suggestions welcome.

[oxarbitrage]

eos reference: http://t1api.eos.io/v1/docs/

[oxarbitrage]

in talks with @wmbutler another option can be swagger(https://swagger.io)

looks more complete than apidocs, demo can be seen at: https://market.mashape.com/musixmatch-com/musixmatch#album-tracks

swagger needs that we add a swagger.json into the core and to add specially formatted comments at the sources files just as apidocs according to @wmbutler .

adding documentation to the .cpp is maybe not a good idea, maybe better to just add duplicated doc at the .hpp for database_api and api.

[wmbutler]

Open API 3.0 is the standard. Swagger is the toolset.

[wmbutler]

@oxarbitrage @ryanRfox I'm inviting Luke Stokes to the conversation. Will one of you please invite him to the Telegram Dev channel? Luke has extremely deep knowledge of API's and recently sold his company. He's also a devoted STEEM and BTS advocate.

[lukestokes]

Hey all. Sorry, I've been a bit busy this week and haven't jumped in yet. I've been chatting with Alex M - clockwork a bit in Telegram to try and get a feel for the overall project. I assume that's the best place to continue discussing the details (though I'll add some thoughts below as well).

I have my opinions on using something like Swagger for a true REST api (Hypermedia API), but in this case it's probably just what you want for RPC stuff. There are also other tools available to translate it into whatever else you want, once you have it in a machine readable format. My understanding at this point is we just need to go from what's listed here and get it into an Open API 3.0 document, yes? I wasn't clear if there'd be additional changes made to the actual APIs themselves and/or if there'd be any redesign of the API surface at all to unify things. The way the APIs are now, would it actually be 5 different documents? I haven't yet personally used the bitshares API, though I have played with STEEM for various reports I've coded, so I do have a learning curve to figure out as well. I'm not sure about the timeline for this or expectation for how quickly things will happen.

Thanks for thinking of me @wmbutler.

[Zapata]

Most of the tools are focused on REST based APIs, not JSON-RPC apis as currently used. There are many issues in Open API 3.0 for JSON-RPC (see OAI/OpenAPI-Specification#801 and OAI/OpenAPI-Specification#664). It's doable using oneOf syntax, but it's hard to maintain and unreadable in most of the tools. Note that someone asked for Steem on their mailing list, he got a negative answer. It should be the same issue with ApidocJS. Unfortunately this kind of issues is one of the main reason poeple now choose REST over JSON-RPC. Note that EOS also have moved from JSON-RPC to REST based API (see doc). Slate is a static website generator from Markdown... it does not parse the code, nor take a schema as input, but is a valid solution. This unknown tool seems to do the job with his own specification format... I can't find any better tool :-(

[abitmore]

Thank you @Zapata.

Frankly speaking I didn't see much difference between EOS REST API and current BitShares JSON-RPC API. The only change is EOS API moved api category and method name to URI and added a version tag, and putting the parameter list directly in a JSON.

The JSON-RPC API in BitShares, e.g. to get a block:

$ curl -d '{"id":1,"method":"call","params":["database","get_block",[1]]}' https://api.bts.mobi/ws

{  
   "id":1,
   "jsonrpc":"2.0",
   "result":{  
      "previous":"0000000000000000000000000000000000000000",
      "timestamp":"2015-10-13T14:12:24",
      "witness":"1.6.8",
      "transaction_merkle_root":"0000000000000000000000000000000000000000",
      "extensions":[  

      ],
      "witness_signature":"1f53542bb60f1f7a653bac70d6b1613e73b9adc952031e30e591e601dd60d493ba5c9a832e155ff0c40ea1dd53512e9f93bf65a8191497ea67d701bc2502f93af7",
      "transactions":[  

      ]
   }
}

The REST RPC in EOS:

$ curl  http://127.0.0.1:8888/v1/chain/get_block -X POST -d '{"block_num_or_id":5}'

{
  "previous": "0000000445a9f27898383fd7de32835d5d6a978cc14ce40d9f327b5329de796b",
  "timestamp": "2017-07-18T20:16:36",
  "transaction_merkle_root": "0000000000000000000000000000000000000000000000000000000000000000",
  "producer": "initf",
  "producer_changes": [ ],
  "producer_signature": "204cb94b3186c3b4a7f88be4e9db9f8af2ffdb7ef0f27a065c8177a5fcfacf876f684e59c39fb009903c0c59220b147bb07f1144df1c65d26c57b534a76dd29073",
  "cycles": [ ],
  "id":"000000050c0175cbf218a70131ddc3c3fab8b6e954edef77e0bfe7c36b599b1d",
  "block_num":5,
  "ref_block_prefix":27728114
}

Seeing this, I think we can:

• perhaps modify a tool to generate docs for our JSON-RPC API;
• implement REST API in BitShares by back porting some code from EOS.

[Zapata]

There are big differences between REST and JSON-RPC, see this stackoverflow question for example, and it's a widely debated subject. Nowadays REST is more popular.

OpenAPI might handle JSON-RPC enventually, it's in their 3.1 wishlist (see "Supporting Message Based APIs"). Maybe AsyncApi is the best choice, it generates documentation either with DocGen or to a Slate compatible markdown using widdershins.

Exposing Bitshares in a REST API style might be nice in the long term, but I don't think it worth the effort for the "API Documentation" subject. It will require to expose both style for a smooth migration as all the clients need to be updated. For reference EOS has a htttp_plugin, wich is used in this way.

[clockworkgr]

As far as "Exposing Bitshares in a REST API" is concerned, I've given some thought to the subject over the last few weeks and I'm starting to think it's something that's better suited to a standalone separate service which would also allow for many extras (from rate-limiting to permissioned access to microcaching etc.) without burdening -core.

Stellar seems to have followed that route with their horizon server.

[abitmore]

Created another issue for discussing REST API support: #870.

[lukestokes]

I'm sorry I haven't been able to be engage in this project as I had hoped. I was added to the eosDAC launch team and have been pretty busy the last two weeks with various exciting things.

I'm certainly a fan of REST over RPC, so I'm glad to see that conversation happening as well.

I messaged with both Kin Lane and Darrell Miller who I greatly respect in this space and there weren't any obvious answers (though gnostic was mentioned). OpenAPI was mentioned, but not as ideal for reasons already described here.

My suggestion at this point would be to work together with existing Graphene communities (BitShares, STEEM, EOS, Peerplays, etc) and try to come up with something consistent between them so developers from one chain could be easily onboard to another. Maybe start a shared Telegram channel for graphene documentation development and design? Based on the financial investment and investor interest in EOS so far, I think whatever they are doing in this space would be worth seriously considering for BitShares also. From a developer's perspective, seeing familiarity in documentation goes a long way. I also think a documentation spec document of some kind (be it OpenAPI or something else) goes a long way in that automated conversions to something different in the future become a possibility.

Ideally, REST (and I mean real rest with Hypermedia as the Engine of Application State, as mentioned in my links above) will be what is standardized on in the future, but I see the need for clear documentation now for those using RPC.

[dwasyluk]

We're dealing with this same dilemma (REST v JSON-RPC docs) on Syscoin. Interestingly enough we built a Swagger spec on top of the Syscoin core with a thin Node/Express layer on top to enable a real REST style API.

We're working on documentation for the JSON-RPC itself now were looking to Swagger for the same thing when we hit the considerations already being discussed. I'm still looking into OpenAPI spec to hopefully support this but the support for OAS3.x out there isn't great yet. Any further updates on where Bitshares ended up with this?

[ryanRfox]

@Zapata will be working this Issue. I've requested he reach out to @wmbutler and @svk31 to receive guidance on what the UI Team may desire. @clockworkgr and @abitmore have provided many comments to this Issue and #870 as well. I'm assigning #870 to @Zapata as they are related and likely progress together.

[Zapata]

You can find an example of what could be done with mzernetsch/jrgen tool there: http://api-docs.dex.trading/.

The approach is very similar to Swagger: a JSON Schema should be produced.

With the schema you can generate htmls docs and other usefull stuff like Markdown documentation, GitBook, NodeJS mock server, ....

The schema could also be used:

• as a specification between UI and core developers, to develop in a "contract first" approach;
• to automate non regression tests of the API.

A nice thing of the HTML doc it's that it shows sample request/responses, and it make testing easy. Any feature could be added as we can create/modify existing "generators" (as I already did).

The code is hosted there.. It's currently a draft.

There is lot of work to document manually the full API, especially get_block and get_object as those return many different types of objects that needs to be modelized using JSON Schema. Maybe it's possible to (partially?) generate the schema from the code. This will speedup the work. I see it as a one shot, as I don't recommend a continuous generation of the documentation from the code as this will break the "contract first" approach.

Let me know your thoughts, so we can see if we continue in this direction.

@ryanRfox If we go this way, there is no link with #870.

[cedar-book]

@ryanRfox, I have this section for API related information. I thought.. @Zapata might be able to use this section to create a page file or a link about any API related information.

I do not know yet what would be the best way to communicate about the documentation with the developers for this repo. But I thought we could start trying this GitFlow?? There are empty temp files, free to use (change/update) in the api_support folder. I've added a page and link for the Explorer and the Wrappers info. Others are sample/note to be updated. Any comments/thoughts would be appreciated.

[TheTaconator]

As I see it, there are three different things being discussed:

(A) how to document the existing RPC API (B) how might the existing RPC API be improved (C) what might be an additional REST API (D) how to produce API documentation for any of these APIs

Regarding (A), I think that embedding specially formatted comments directly above a method in the code is the most productive because the developer is already looking at the code and can make any adjustments more easily. It is also obvious when this pre-cursor to the documentation is missing. Using another file is more error-prone but maybe if the output results (D) are so much better then the additional work might be worth it.

Regarding (D), apidocjs appears to do a fine job for the existing RPC API. If the highest priority is to create technical documentation, which is neither a tutorial nor a guide, of the existing API then this impresses me as the simple and fast way to proceed.

[TheTaconator]

The schema could also be used:

as a specification between UI and core developers, to develop in a "contract first" approach; to automate non regression tests of the API.

A nice thing of the HTML doc it's that it shows sample request/responses, and it make testing easy. Any feature could be added as we can create/modify existing "generators" (as I already did).

@Zapata The contract between the UI and core developers is an important issue. Could you please elaborate on how the testing might be made easier if using the schema while also considering that the current RPC code is implemented in C++? Are the tests that you are envisioning written for execution in JS and referencing an actively running RPC node on either a public or private blockchain?

[Zapata]

Regarding (D), apidocjs appears to do a fine job for the existing RPC API. If the highest priority is to create technical documentation, which is neither a tutorial nor a guide, of the existing API then this impresses me as the simple and fast way to proceed.

The issues with apidocs is that it currently handles only REST style. This means we need to patch it to handle JSON RPC style. This might be possible through the plugin mechanism. Then I don't know if apidocs can handle all the complexity of the Bitshares model (especially class hierarchy), where I know Json Schema can. This might be harder to solve.

A separate schema file allow UI developpers and BSIP writers (without C++ knowledge, nor bitshares core code structure knowledge), to propose changes to the API in a formal way: a diff of the schema change, and could be included in specifications/bsips.

Also Json Schema is used by jrgen and swagger and is more popular (standard?) than apidocs annotation. However an apidoc plugin exists to define parameters and result types in Json schema, and another exists to generate json schema from apidocs annotations.

May be we should try to make apidocs works with JSON RPC if the approach is preferred over jrgen.

Could you please elaborate on how the testing might be made easier if using the schema while also considering that the current RPC code is implemented in C++? Are the tests that you are envisioning written for execution in JS and referencing an actively running RPC node on either a public or private blockchain?

With a parsable schema that contains sample values we can programatically load it and generate all the api calls against a running RPC node (public or not). We can then validate the returned data structure using schema, or better if we use a "snapshot" node (stopped at a specified block number) validates the data returned (using the sample values).

This will make sure that api documentation and code are always in sync, and may trap unwanted changes/regressions.

The tool could be written in any language that can parse the schema:

• for apidocsjs it's javascript only through the core library
• for jrgen it's any language that can parse JSON.

edited: You can also generate a Stub server easely with the schema, usefull for UI developers.