[CSL-164] Servant API for node

[jagajaga]

@manpages

As there are engineers coming onboard to work with PureScript in order to start integration of pos-implementation and Daedalus wallet, we have to refactor state and provide Servant endpoints akin to RSCoin endpoints that were exposed for Block Explorer. Plus provide means to expose RPC calls to be called from Daedalus Wallet.

# Parent for: #590, #596

[jagajaga]

@georgeee

Before doing, please list all API here And estimate 1d for whole API seems to be too little

[jagajaga]

@jagajaga

@gromak ask me for all necessary info

[jagajaga]

@manpages

@georgeee that's @jagajaga's estimate, my estimate was 2d for @gromak, let's see who was closer :)

[jagajaga]

@manpages

** TODO URGENT: Servant API Fourth Categories

<2016-11-28 Mon> *** Endpoints that work with big data sets, block-explorer type of things (NOT SURE IF NEEDED FOR THE CURRENT DESIGN, MUST ASK DESIGN TEAM) *** Endpoints AND changes to the State on the backend to reflect Node state and select interesting transactions for the current user *** Endpoints that provide basic cryptocurrency use cases — sending txs, picking scripts (WILL NOT BE COMPLETE UNTIL ARTYOM IS DONE WITH PLUTUS INTEGRATION) *** Endpoints that provide enough data about staking for the current version of staking UI and a toggle for participation in staking @georgeee @gromak @jagajaga if you have any questions — ask.

[jagajaga]

@gromak

@jagajaga @manpages @georgeee Two questions:

1. Where should all these endpoints be? Currently we have only one stateful entity which is full node. Do we need to introduce more entities? For example, for block-explorer-like endpoints we may introduce, well, block-explorer, who will store some state and this state will have some extra information (not needed by full node, but needed by block-explorer, just like in rscoin) and it won't have some information stored by full node (e. g. SSC data). Good thing is that we don't add extra mess to full node and don't add useless data to block-explorer, but bad thing is that some data will be duplicated if we run block-explorer and full-node on the same machine. Regarding ''endpoints AND changes to the State on the backend'': which state are we talking about? Is it state of full node or state of wallet? Or do we want to have wallet capabilities on full node? For example, in haskoin, AFAIK, these two things are separated. Do we want to separate wallet from full node? Regarding ''endpoints that provide basic cryptocurrency use cases'': it's related to previous question, currently node doesn't create transactions, she only receives them and puts into blocks. Regarding ''Endpoints that provide enough data about staking for the current version of staking UI and a toggle for participation in staking'': this functionality is part of full node, so they can be added to full node and this part is more clear. ——————————————— So I would like to know for which questions we already know answers, which questions should be discussed, which questions I should answer by myself. Currently I don't see any advantages in separating node state and wallet state (because apparently wallet needs everything that node needs, except optional SSC data which can be turned off). But maybe you have something to say about it. With block explorer it's less obvious, because block explorer doesn't really need to process transactions/SSC, for example. ——————————————————————————————
2. Should these endpoints be very generic (so that the same API can be used for different reasonable cryptocurrencies) or ouroboros-specific? For example, should they be similar to endpoints we have in RSCoin? I know that set of endpoints in RSCoin is very small, but anyway.

[jagajaga]

@georgeee

1. I think we can make it a pluggable part of node (turning it on/off with CLI key)

[jagajaga]

@gromak

2. I know this question is obscure, so you can ignore it if it doesn't make sense.

[jagajaga]

@gromak

1. I think we can make it a pluggable part of node (turning it on/off with CLI key) Alternative solution is to make it a separate project which will interact with cardano-sl using main protocol over TCP and will have cardano-sl library as dependency.

[jagajaga]

@gromak

Now I am putting it into Pos.Web, but I would like to have answers ASAP, so that I don't have to redo everything.

[jagajaga]

@gromak

Btw, do I understand correctly that staking means participation in SSC?

[jagajaga]

@gromak

Currently I don't see any advantages in separating node state and wallet state (because apparently wallet needs everything that node needs, except optional SSC data which can be turned off). At the same time, wallet needs more than full node needs. For example, wallet needs to track own transactions to report status of them and resend if necessary.

[jagajaga]

@gromak

Do we already know how authentication should be done? Probably it depends on what frontend can do.

[jagajaga]

@georgeee

Alternative solution is to make it a separate project which will interact with cardano-sl using main protocol over TCP and will have cardano-sl library as dependency. Just launch node with key that has empty stake. Will be just the same :)

[jagajaga]

@gromak

The question is whether web server should share database and all node's state with node itself or it should be another project operating on state via some lower-level RPC or something like that. I assume the former.

[jagajaga]

@manpages

Btw, do I understand correctly that staking means participation in SSC?

Yes

Separation of node and wallet state is reasonable. Remember that wallet works with aggregates of information about the node state, so why don't we put it elsewhere and recompute those potentially with lower priority than protocol-level stuff?

block-explorer

Forget about it for now, the most important thing is to get endpoints that feed node-specific information to the wallet and that expose basic cryptocurrency functionality implemented.

so that we won't have to redo it

This work starts the practical conversation about the structure of universal wallet API. The further work will be done by Lars Kraus, the PureScript guy we've hired. So don't overthink those bits, just get the data served over HTTP and write functions that derive the necessary data from the state.

@gromak @jajagajaga @georgeee understood?

[jagajaga]

@gromak

Separation of node and wallet state is reasonable. Remember that wallet works with aggregates of information about the node state, so why don't we put it elsewhere and recompute those potentially with lower priority than protocol-level stuff?

Ok, somewhere inside I totally support this separation even if benefits are not absolutely clear now. AFAIU, it means that we will have cardano-node (which is what we have now but with optional HTTP server which can be enabled from CLI and provides, amongst others, endpoint to toggle SSC participation) and separate executable cardano-wallet (which is another HTTP server and provides, amongst others, endpoint to submit transaction). Is it correct?

On the lower level, cardano-node's web server locates in Pos.Web in cardano-sl library and shares states of node with main code and operates on it directly. Regarding cardano-wallet, it's less clear, we can even make it a separate project (which depends on cardano-sl library) or we can make it another executable inside cardano-sl. If we make it another executable we can put all logic into this executable or partially into cardano-sl library. By default I will put it into cardano-sl library (e. g. Pos.Wallet) because usually when we put too much logic into executable we end up moving this logic into library sooner or later. Let me know if it's bad.

Regarding authentication: I assume that now I don't have to think about it and that if cardano-wallet receives request to send coins from wallet to someone else this request is sent by owner of wallet.

[jagajaga]

@gromak

cardano-node's web server locates in Pos.Web in cardano-sl library and shares states of node with main code and operates on it directly. At the same time nothing in Pos depends on Pos.Web. Only executables know about Pos.Web.

[jagajaga]

@gromak

Currently we have this API:

type BaseNodeApi ssc =
    "current_slot" :> Get '[JSON] SlotId :<|>
    "leaders" :> QueryParam "epoch" EpochIndex :> Get '[JSON] SlotLeaders :<|>
    "spending_key" :> Get '[JSON] PublicKey :<|>
    "head_hash" :> Get '[JSON] (HeaderHash ssc)

We also have GodTossing specific API:

type GodTossingApi =
    "toggle" :> Capture "enable" Bool :> Post '[JSON] () :<|>
    "secret" :> Get '[JSON] LSecret

If node is launched with GodTosssing SSC she uses the following API:

type GtNodeApi =
    BaseNodeApi SscGodTossing :<|>
    "god_tossing" :> GodTossingApi

otherwise BaseNodeApi is used. Everything is implemented except god_tossing/secret endpoint. It depends on CSL-219. Under this issue I want to implement this endpoint and then I suppose it can be closed. These endpoints cover 2-nd and 4-th categories. If we need more endpoints from these categories, we can create new issues for them and adding these endpoints should be easy (because all infrastructure for it is already written). 3-rd category of endpoints is not started yet, it depends on CSL-214. I will create a separate issue for it because I like more fine-grained issues. I will also create a separate issue about documentation of API.

[jagajaga]

@gromak

Currently we have these APIs with all endpoints implemented:

type BaseNodeApi ssc =
    "current_slot" :> Get '[JSON] SlotId :<|>
    "leaders" :> QueryParam "epoch" EpochIndex :> Get '[JSON] SlotLeaders :<|>
    "spending_key" :> Get '[JSON] PublicKey :<|>
    "head_hash" :> Get '[JSON] (HeaderHash ssc) :<|>
    "local_txs_num" :> Get '[JSON] Word

type GodTossingApi =
    "toggle" :> Capture "enable" Bool :> Post '[JSON] () :<|>
    "has_secret" :> Get '[JSON] Bool :<|>
    "secret" :> Get '[JSON] SharedSeed :<|>
    "stage" :> Get '[JSON] GodTossingStage

I think it should be enough to play with this API, at the same time I understand that it's not exhaustive. I don't want to guess which endpoints will be needed by frontend, instead I would prefer to add them lazily, when someone really needs them. Infrastructure for doing it is written.

To put it all together, I suppose that first version of API for categories 2 and 4 is ready. The only missing part is documentation, it will be done as CSL-224. Category 3 is not ready, it will be done as CSL-223. Category 1 is postponed.

This issue can be closed.