Discussion: tarpc (RPC layer) is rust specific.

[dan-da]

related to #6.

I was surprised to find out that tarpc is a rust-specific RPC system, which of course would mean that all clients either need to be written in rust, or develop a tarpc impl for that language. I think this would be shooting ourselves in the foot when it comes to getting exchange integration, onboard 3rd party wallet developers, etc. Maybe this has already been considered/discussed?

I get that tarpc is slick, easy to implement, and provides some nice features like auto generation of an RPCClient from a trait, and full traceability between client/server.

My thought is that perhaps we can keep tarpc eg for the dashboard and future "official" wallet(s), but also add jsonrpc for compatibility with everything else. Hopefully the jsonrpc would just be a wrapper around the existing tarpc API functions. Or maybe those funcs will need an rpc abstraction layer.

jsonrpc of course has its warts. It is pretty verbose, for one. But it has also been kind of lingua-franca amongst cryptocurrencies and exchanges. It is also very simple to write by hand if need be, and debug requests/responses manually, a nice property. So I would think that we could have the jsonrpc for compat and easy use, and then tarpc is available for anyone that needs better performance.

Of course any cryptocurrency written in rust would face similar decisions so it may be worth surveying what others have done. My information may be out-dated. And I do notice that the rust jsonrpc crates haven't been updated in a couple of years which is not a good sign.

[Sword-Smith]

I was also not aware that tarpc is a Rust-specific RPC system. I believe this choice comes all the way back from when @Munksgaard helped me create a skeleton for the client. I agree with your sentiment that the bar for integrating Neptune should be lowered as much as possible. Maybe some data structures will not function with jsonrpc though? From what I recall, JSON can only handle structs with named fields and lists natively, not fancier data models like tuples, hash maps, enums etc.

[dan-da]

Well that's the thing with a lingua-franca right, is that you tend to get least common denominator. But let's see:

• hash map is basically just a dict {key: val} in json
• tuple is basically a json array, which supports mixed types, though perhaps best to avoid mixing in RPC apis
• rust style enum doesn't seem to have a good mapping, but then I don't normally see enums used in RPC apis

So yeah the jsonrpc wrapper layer would have to some mapping. But actually serde-json can do most of that. I've used it before with good results.

Also, we could potentially offer an alternate serialization in eg RON (rust-object-notation) that supports all rust types. Maybe redundant if we keep tarpc though.

Where I'm uncertain is that it's possible that by now the cryptocurrency industry has shifted to some other multi-lang RPC standard. Eg, what do ethereum, solana, cardano, tezos, zcash use? I'll check into those a bit.

[dan-da]

Methodology:

I googled <cryptocurrency> rpc api for each cryptocurrency and checked results for jsonrpc or anything other than jsonrpc.

Findings:

Bitcoin and clones, including zcash - jsonrpc https://en.bitcoin.it/wiki/API_reference_%28JSON-RPC%29 https://zcash.readthedocs.io/en/master/rtd_pages/payment_api.html

Ethereum - jsonrpc. including ERC-20. https://www.blockchain-council.org/blockchain/a-guide-on-ethereum-rpcs-methods-and-calls/

Solana - jsonrpc. (via http, websocket, web3.js) https://docs.solana.com/api

Cardano - confusing!! json-rpc for sidechains. maybe some REST somewhere. and custom p2p? see: https://www.reddit.com/r/CardanoDevelopers/comments/mavw2h/rpc_calls_and_cardano/ and: https://docs.cardano.org/cardano-sidechains/example-evm-sidechain/api/

Tezos - REST/json https://tezos.gitlab.io/active/rpc.html

Monero - jsonrpc and REST/json https://www.getmonero.org/resources/developer-guides/daemon-rpc.html

Algorand - REST/json. OpenAPI https://developer.algorand.org/docs/rest-apis/restendpoints/

Polygon - jsonrpc https://wiki.polygon.technology/docs/pos/reference/rpc-endpoints/

Cosmos - Protobuf/gRPC, REST/json, cometBFT https://docs.cosmos.network/main/learn/advanced/grpc_rest

Observations:

It seems the majority are still using jsonrpc and even the ones that use REST are still returning json.

Most that use REST/json define the APIs with OpenAPI. At least one of the jsonrpc users provides an open-rpc interface definition.

The cardano reddit page makes some statements about RPC calls being bad for node scalability, which might be worth considering further, and look into their solution.

Also, some have versioning for their APIs, which we should perhaps think about.

[dan-da]

I think there may be a nice solution possible, where we can keep tarpc and also get json-rpc compatibility.

I was reading the tarpc docs a bit, and it is actually transport-agnostic. ie, it supports pluggable transports.

There are presently transports for serde, in-memory channels, bincode, and json. All of these implement the tarpc Transport trait, which is basically a bidirectional stream of bytes.

I did not find any json-rpc transport, but hopefully it would not be too difficult to write one. The idea would be to take an existing jsonrpc crate and impl the Transport trait for it.

There might be unforseen difficulties, but it seems plausible.

Alternatively, we might just want to use tarpc_json_transport . I notice we are already using serde's JSON serializer, so I'm unclear what the difference is between that and tarpc_json_transport.

Regardless of transport, for cross platform compatibility, we will likely need to be careful to use only basic data structures like struct and array. complicated enums wouldn't be supported by most languages, for example.

lastly, I'm unsure if tarpc supports serving multiple transports at the same time, or one must pick just one. If not, we could probably start multiple RPC server to have one transport per port. It might make sense to have eg json-rpc that is slow but cross-platform and human friendly and also bincode or similar that is very fast.

anyway, I am encouraged by this discovery.