Decimal precision

[ghost]

Issue

The decimal precision can't be too high otherwise the exchange API will reject the request.

eg: When using binance API, if I want to buy 123.45 USDT value of BTC at 9,500.01, the request will be:

price: 9500.01
qty: 0.01299472316

But binance API only accepts 8 decimal on the BTC asset for this trading pair., the request should be converted to:

price: 9500.01
qty: 0.01299472

Proposed solutions

The exchanges provides a list of the pairs with the precision expected. coinbase binance ftx as you can see some exchange are using quote and price precision others are using a price and size increment.

Solution 1

When creating an order, the exchange SDK will fetch the exchange information (from a cache) in order to get the required precision/increment information and apply the rounding/truncating. This has some performances implication when you have to look for a specific pair every times you want to create a new order especially when you are using an exchange with a lots of pairs like binance (881 now).

Solution 2

Create a new type for pair that replace String that will keep the price and qty increments for the pair.

pair_name: String,
price_increment: Decimal
qty_increment: Decimal

An exchanges trait will be added in order to create the pair type, the implementation will look for the exchanges information to get the increments values.

When creating an order the price and qty increments will be fetched from the pair type.

Usage

I suggest adding a new parameter to the OrderRequest types named precision_rule_qty and precision_rule_price with a default value set. For a sell price precision will be defaulted to Round, for buy it will be Truncate, qty will default to Truncate.

The precision rule could also be set on the exchange instead of specified on every order.

enum PrecisionRule {
 Round,
 Truncate
 None
}

Notes

Binance

After trying to submit some orders, they are rejected because they should also fit the step size filter, step size seems to be a better indicator of the precision.

{"symbol":"BTCUSDT","status":"TRADING","baseAsset":"BTC","baseAssetPrecision":8,"quoteAsset":"USDT","quotePrecision":8,"quoteAssetPrecision":8,"baseCommissionPrecision":8,"quoteCommissionPrecision":8,"orderTypes":["LIMIT","LIMIT_MAKER","MARKET","STOP_LOSS_LIMIT","TAKE_PROFIT_LIMIT"],"icebergAllowed":true,"ocoAllowed":true,"quoteOrderQtyMarketAllowed":true,"isSpotTradingAllowed":true,"isMarginTradingAllowed":true,"filters":[{"filterType":"PRICE_FILTER","minPrice":"0.01000000","maxPrice":"1000000.00000000","tickSize":"0.01000000"},{"filterType":"PERCENT_PRICE","multiplierUp":"5","multiplierDown":"0.2","avgPriceMins":5},{"filterType":"LOT_SIZE","minQty":"0.00000100","maxQty":"9000.00000000","stepSize":"0.00000100"},{"filterType":"MIN_NOTIONAL","minNotional":"10.00000000","applyToMarket":true,"avgPriceMins":5},{"filterType":"ICEBERG_PARTS","limit":10},{"filterType":"MARKET_LOT_SIZE","minQty":"0.00000000","maxQty":"526.58296769","stepSize":"0.00000000"},{"filterType":"MAX_NUM_ORDERS","maxNumOrders":200},{"filterType":"MAX_NUM_ALGO_ORDERS","maxNumAlgoOrders":5}],"permissions":["SPOT","MARGIN"]},

Nash

query ListMarkets {
  ListMarkets{
    aAsset{
      blockchain
    } ...
  }
}

Exchanges

This exchanges supports getting the market pairs with the price/qty increment

• okex
• huobi
• kraken

[auterium]

Hello there!

I know the current "market standard lib" (that shall not be named) approaches the market info load as stated in Solution 1 by always checking the cache of the markets for the exchange and loads the data lazily by calling the exchange's market info endpoint when the cache is empty. I think this is a valid approach that could be enhaced by providing the lib user a mechanism to "warm up" that cache (i.e. loading the data from a local file at bootstap instead of doing the API call at runtime).

The enum sounds like a reasonable option for amounts (although not sure about the default to Round) but it should be separate from the rounding of prices. The desired behavior of the price rounding will very likely vary depending on the side of the book where the order is being placed.

[ghost]

Updated the issue with the precision_rule_qty and precision_rule_price.

[jnicholls]

I thought about this for a while. Assuming that the goal of the library is to be perpetually accurate about a pair's precision for a long-running trading application, the pair information for an exchange (which includes its precision) must be kept up-to-date all of the time, using a technique such as a central cache (lookup table). That much is for certain and we all seem to agree on that premise.

I believe Solution 2 would be the most efficient and type-safe approach, allowing a pair to be looked up ahead of order activity being submitted. What would be returned to the user is a handle to the pair named e.g. TradePair or something, which has immutable references to the precision information located in a central cache. This TradePair would be submitted with orders. The library would perform a read-only lookup of the precision information located in the central cache, any time it needed that information (i.e. validating an order prior to submitting it). The library would offer a mechanism to refresh exchange information, which would include precision information for all pairs on an exchange. When this mechanism discovers an update to a pair's precision info, it would take a read-write lock on the info, update it, and release it. All outstanding handles to the pair info would reference the newly updated information.

The above to me sounds like the best approach for performance, type-safety, and perpetual accuracy of exchange information in general, including a pair's precision information. It avoids the hash table lookup cost on every order submission, which when amortized over time is a lot more expensive than a RwLock where there are far more reads than writes.

I can write an example of this data model and share it here for visual clarity.

[ghost]

Challenging approach. I would suggest to start without the update mechanism.

[jnicholls]

The update mechanism is very straightforward. The whole model is straightforward in fact.

[jnicholls]

Here's a basic implementation of the idea. https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=803c29d1f47f513d9e493fd39ac5a09a An improvement to this would be in the refresh() logic, to first do a read() lock on the pair to inspect if anything has changed about it...if not, continue to next pair...if so, upgrade the read lock to a write lock and then do the update. This improvement would avoid any stalls on pairs in use if there were not any updates to their info from the exchange, which is the majority case.

[auterium]

@jnicholls when will this be loaded? At bootstrap or at first request? Keep in mind that if it's done at bootstrap, this could mean doing so for 10, 20, 50, 100 exchanges, depending on the implementor's include pattern and the extension of this library. Otherwise this would need some form of lazy loading (as the leading library does).

The challenge is also that some exchanges have dynamic precision based on a rule.

Bitfinex's pricing precision model is based on 5 numerical places, meaning that if BTC's price is below 10k, rounding is to 1 decimal place, but the moment the price is 10k but below 100k, the rounding is to the dollar (integer), but when it crosses 100k the increments are done in 10 dollars now.

I think that the following are sensible requirements/features:

• Lazy loading the markets info
• Being able to manually load the data (warm up the cache) to avoid the network call
• Per-exchange rounding rules
• Overriding rounding rules
• Refresh/reload option. Refresh meaning it performs another network call to the info endpoint while reload means the implementor can pass a new set of values

One extra note, though. Using a string name for the pair has given us lots of nighmares and it's hard to use for more advanced stuff. A better option is to have the base & quote as separate fields, as it allows for better management & pairing with individual coins (i.e. matching balances to markets) or even mixing markets (multi-market arbitrage).

[ghost]

One extra note, though. Using a string name for the pair has given us lots of nighmares and it's hard to use for more advanced stuff. A better option is to have the base & quote as separate fields, as it allows for better management & pairing with individual coins (i.e. matching balances to markets) or even mixing markets (multi-market arbitrage).

How do you deal with pairs that are more complex like futures or products like btcbull?

[auterium]

That's a fair question, but I've never worked with other than spot markets, so I don't know how futures or btcbull work. I'm not saying it must be something complex, but having like:

struct Pair {
  base: String,
  quote: String,
  base_increment: f64,
  quote_increment: f64
}

Makes things much easier than having a single field for the pair name.

A particular example comes to mind: pair/symbol format for API calls. Doing calls to the ETH/BTC market is different per exchange:

• ETHBTC (Binance)
• tETHBTC (Bitfinex)
• eth_btc (Bitso)

But they are all the same market, so as a library user I would like to say "I want to trade on ETH/BTC both in Binance and Bitfinex" but I don't care how the underlying API call is formed.

Would it make sense to separate models by product type (spot, funding, futures, etc)?

[jnicholls]

I agree with using separate base and quote properties for describing a market. The example was simply to showcase the mechanics of shared handles to centrally manage the exchange market information, including the precision details.

[ghost]

Got a first version on master now. The first approach rounds size using BankersRounding

Price will be rounded up when selling and down when buying.

https://docs.rs/rust_decimal/0.11.1/rust_decimal/enum.RoundingStrategy.html

[auterium]

This is a great start! I wasn't aware of that crate, which looks quite nice. Not sure if I've mentioned this before to you, but there's a particularity on Bitfinex. Right now it's not a biggie, but if an asset goes over 100k, the rouding strategy is no longer based in decimal places, but it will be in increments of 10.

[ghost]

I think that should be ok Decimal library should be able to handle it.

[auterium]

@steffenix are you sure? I agree the decimal library works pretty well when you know in advance the amount of decimal places to use, and this will rarely vary on an individual market. The problem with Bitfinex is whenever the orders of magnitude change, the amount of decimal places is reduced. I.e. on [1000, 10000) the decimal places is 1, while [10000,100000) is 0 and then, when reaching [100000,1000000) due to their significant figures strategy, now the "price steps" will be 10, instead of 1 (or a -1 decimal places)

Lets say that the current top bid is 9999.9 and top ask is 10000, meaning a mid-price of 9999.95. If I want to place my sell order at 0.01% from the mid-price, I would have to round to 0 decimal places, while the buy order would need a rounding of 1 decimal place. I'm struggling to see how the rust_decimal crate could be used in this scenario

[ghost]

Understood Bittfinex isn't part of the scope now. That can be tough when we have to add it.

Is that logic exposed as part of an API endpoint?

If it's always the same logic with the same precision (5) we would be able to have a custom conversion mechanism for bitfinex.