Feature: CEX-like JSON API

jgarzik commented 3 years ago

Summary

Add an HTTP/JSON trading API alongside the existing gRPC API.

With the exception of authentication, the API should mirror [one of: Bittrex/Coinbase/Binance] trading API, for maximum compatibility with existing trading systems and bots.

Motivation

Bring more trading and liquidity to OpenDEX, via 98%-100% compatibility with an existing, popular trading API. The popular Hummingbot trading/MM bot already knows these APIs, and can even be connected to a Hummingbot liquidity mining program, bringing more liquidity to OpenDEX.

Recommended configuration

A TCP port is exposed, to which external trading systems may connect. Docker & xud configuration will need settings related to this.

Recommended APIs

(Summarized from bittrex, coinbase and binance APIs)

Read-only market APIs

Get list of trading limits
Get list of markets (list pairs)
Get market(s) summaries: last 24 hours volume / # change / high / low / last-trade-time
Get market(s) tickers: last-trade, best bid, best ask
Get market's order book: array of bids, asks
Get market's list of recent trades
Get recent/historical candles

Private / User Trading

Get my closed orders
Get my open orders
Get metadata for specified order
Cancel order
Cancel all orders
Create order

Private / User Wallet

Get trading balances. Show locked (in orders) and unlocked balances.
Get wallet balances.
Move funds from wallet to trading.
Move funds from trading to wallet.
Deposit process: Get deposit addresses, etc.
Transfer process: Transfer funds to another L2 entity.
Withdrawal process: Transfer funds to another L1 on-chain entity.

Implementation notes

In theory, a proxy can do this. However, a proxy solution is more complex than a built-in API, across a number of vectors.

erkarl commented 3 years ago

@sangaman maybe we should look into reviving the gRPC <-> REST proxy?

https://github.com/ExchangeUnion/xud/blob/master/sample-xud.conf#L86

I haven't tested this in a long time, but it used to work.

kilrau commented 3 years ago

Not a small feat, but would definitely increase the reach for existing trading bots, like Hummingbot. I also know that quite some bots are written against ccxt. I am wondering if anybody ever tried to make hummbingbot work with ccxt after this.

https://github.com/ccxt/ccxt https://github.com/ccxt-rest/ccxt-rest

We discussed ccxt compatibility way back. Here the list of unified ccxt calls.

erkarl commented 3 years ago

Thanks @jgarzik for getting the ball rolling on this one.

kilrau commented 3 years ago

We just discussed to revive our existing webproxy in POC style for you to evaluate if that behaves how you would expect it to behave (streaming calls are not interesting for you as far as I understand it nor does hummingbot seem to use streaming calls!?). Once you have a look at that and check on the vectors that you mentioned, we can decide if it's sufficient or if we want a native JSON API.

We can discuss the API structure and call list in a second step.

If a webproxy doesn't make sense, please step in @jgarzik

jgarzik commented 3 years ago

@kilrau 1) web proxy is a workable solution, yes. Happy to test. This would be great.

2) Web proxy solution has the advantage of being build-able/maintain-able separate from the core daemon. Web proxy solution has the disadvantage of often "falling behind" the main node software, lagging behind in terms of both features and bug fixes.

3) The "CEX-like API" should include websocket streaming support to fully support modern traders. I did not mention this in this issue, to avoid over-filling the issue with even more detail. JSON is step 1, the base level of support.

4) The "CEX-like API" should include FIX protocol support to fully support modern traders. I did not mention this in this issue, to avoid over-filling the issue with even more detail. JSON is step 1, the base level of support.

erkarl commented 3 years ago

@jgarzik

web proxy is a workable solution, yes. Happy to test. This would be great.

Looks like the webproxy solution that we have right is working with the exception of streaming calls. There are some bugs to iron out, but this should give you a good overview what the proxy will look like if we choose to go this route.

It can be enabled in the xud.conf:

[webproxy]
disable = false
port = 8080

The documentation will then be available at: http://localhost:8080/api-docs/ Api root URL: http://localhost:8080/api

Example call for balance

➜  xud git:(master) ✗ curl -X GET "http://localhost:8080/api/v1/balance" -H  "accept: application/json" | jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   587  100   587    0     0   1304      0 --:--:-- --:--:-- --:--:--  1301
{
  "balances": {
    "BTC": {
      "total_balance": "46554492163",
      "channel_balance": "8000950",
      "pending_channel_balance": "0",
      "inactive_channel_balance": "0",
      "wallet_balance": "46546491213",
      "unconfirmed_wallet_balance": "0"
    },
    "ETH": {
      "total_balance": "2999963865",
      "channel_balance": "0",
      "pending_channel_balance": "0",
      "inactive_channel_balance": "0",
      "wallet_balance": "2999963865",
      "unconfirmed_wallet_balance": "0"
    },
    "USDT": {
      "total_balance": "200000000000",
      "channel_balance": "100000000000",
      "pending_channel_balance": "0",
      "inactive_channel_balance": "0",
      "wallet_balance": "100000000000",
      "unconfirmed_wallet_balance": "0"
    }
  }
}

kilrau commented 3 years ago

What's your take on the current state of the webproxy? @jgarzik

jgarzik commented 3 years ago

@kilrau Will check it out. Primary task is a side-by-side comparison of this vs Binance/Bittrex/coinbase, which is largely a documentation effort (vs testing/coding).

ExchangeUnion / xud