Open andre77 opened 5 years ago
I like the idea. Maybe we can add options support also. They just need expiration date field and strike price i believe.We can create a second constructor for option support. What do you thing?
futures also have expiration date, i do not mind to add an optional Date field, which would be null for 'spot' and 'perpetual' products
Great.
On Tue, 13 Aug 2019, 11:40 André Reiter, notifications@github.com wrote:
futures also have expiration date, i do not mind to add an optional Date field, which would be null for 'spot' and 'perpetual' products
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/knowm/XChange/issues/3171?email_source=notifications&email_token=AHIWQ7YH5XMMRK7CMZLGXCDQEJXPFA5CNFSM4ILH5FDKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4E63VQ#issuecomment-520744406, or mute the thread https://github.com/notifications/unsubscribe-auth/AHIWQ747P7NWYHQWS2AD7YDQEJXPFANCNFSM4ILH5FDA .
sounds good to me.
Have anyone implement this or at least started?
not yet unfortunately
@andre77 wondered if you had any thoughts on the product PR idea, just I was going to look at implementing TFX which would need a simliar solution, I was going to approach it a bit different but may be your way is better.
no, unfortunately did not find any time to make some progress here
Maybe we can add options support also. They just need expiration date field and strike price i believe.We can create a second constructor for option support.
Vanilla options can be covered this way, but any Complex option will need more (for instance, is it American, European or Asian style (multiple expiry dates)), and there are Binary options etc... It may be better to add a hashMap to the Product to support all the permutations than can be populated from ExchangeMetaData (local or downloaded).
I am not sure we want to build a normalized product database here, but allowing products to have un-restricted properties would be a great halfway house.
see Month Codes, for instance Z for December might be useful for sorting. These codes refer to the 3rd Friday in the month, but I see from the data above BTC-27DEC19 and that is the 4th Friday, expiry date needs to be kept.
how about we introduce an abstract instrument class, which the existing Currency pair extends, then each exchange implentation can extend the instrument class as they needed for any particular exchange implentation weather that be using month codes, strikes, option types, expiration date. just think this would reduce the amount of refactoring.
@andre77 @makarid @timmolter @walec51 looking at this a bit more may be abstract classes say 'Instrument' with two concrete classes of 'CurrencyPair' and 'Contract' might create more hassle in the JSON serialisers that using concrete classes. To avoid a large refactoring what are the thoughts of creating a 'Contract' class that extends 'CurrencyPair'. This approach implicitly assumes any derivative/contract would be FX style with a base and quote currency?
I would avoid extending CurrencyPair, and wrap it instead. Because some contract will be defined by multiple CurrencyPair objects.
Sure, so may be I will take @andre77 original suggestion and introduce a new concrete class say called 'instrument' that wraps currency pair and update all the interfaces to use new instrument class.
Sure, so may be I will take @andre77 original suggestion and introduce a new concrete class say called 'instrument' that wraps currency pair and update all the interfaces to use new instrument class.
We could first create a new interface Future.class and sub divide it to FutureAccountService,FutureTradingService,FutureMarketDataService and implement these to the exchanges that support futures. That way we could create the Instrument.class and only add it the methods of these interfaces. That way we don't need to modify the existing classes. What do you think?
'Instrument' with two concrete classes of 'CurrencyPair' and 'Contract' might create more hassle in the JSON serialisers that using concrete classes
Jackson can handle far more complex class hierarchies then this - you just have to know how to configure it via annotations - this is not a problem
FutureAccountService,FutureTradingService,FutureMarketDataService
would those interfaces be any different from the normal interfaces other then from the fact that they accept Future instead of CurrencyPair?
what future specific methods would there be in a FutureAccountService?
what future specific methods would there be in a FutureMarketDataService?
do exchanges expose totally different APIs for futures?
do exchanges expose totally different APIs for futures?
Yes, frequently (and they often replicate the spot API which changes for Futures) for instance https://support.kraken.com/hc/en-us/categories/360001806372-Futures-API
Also, they can use different endpoints
what future specific methods would there be in a FutureAccountService?
what future specific methods would there be in a FutureMarketDataService?
I don't know the exact methods yet. We could add them when we will try to implement at least one Future exchange in order to see what we will need. I was just posting about an idea of how we could solve the issue of adding Future support but without break things from the spot market exchanges. Definitely we would need at least the following methods which there aren't in existing Xchange implementation(we can change the names of course):
getOpenPositions getCurrentMarginUsed
Where would leverage go? on the trade and position rather than the Instrument? Maybe on the instrument to if leverage is fixed for that symbol.
Also, how to handle generic futures (March,June,September,December) vs dated contracts expiry (2020-03-27)? Send in a trade for March-2020 get back executions for 2020-03-27?
Where would leverage go? on the trade and position rather than the Instrument? Maybe on the instrument to if leverage is fixed for that symbol.
Also, how to handle generic futures (March,June,September,December) vs dated contracts expiry (2020-03-27)? Send in a trade for March-2020 get back executions for 2020-03-27?
There are 2 types of leverage. The instrument leverage(maximum leverage supported) and the openPosition leverage that is a real-time metric in order to see how much leverage your position is using.
I didn't understand the second question
The question that i would like to ask. Do we need to create a Position.class that extends Order.class or it is better to not extend Order.class? I am asking this because a Position is very different than an Order. A position's metrics(margin,value) changing overtime but this is not true with an Order. Also, i believe that we should make this issue top priority because it opens the "world" for more exchange implementations which are not possible right now. What do you think?
So, as i see it right now:
FutureAccountService.interface : contains { getAccountUsedLeverage() } FutureTradingService.interface : contains { getOpenPositions(), getOpenPositions(CurrencyPair currencyPair, getFundingHistory(CurrencyPair currencyPair} Also we need to add an OpenPosition.class.
What do you think? I will start the implementation when we all agreed on something.
I would not recommend extending the Order Class, I think it has already been overused (Trades and OrderBook) and the LimitOrder MarketOrder, StopOrder structure is not ideal.
Generic futures codes are aliases to the nearest expiry month
There are so many ways this could be done, which makes it hard to decide. We could add new DTOs, Services and methods. We could add new DTOs and methods to existing services. We could add new and/or extended DTOs to existing Services and existing methods.
I always tried to lean towards making it easy for the end user (a programmer using the library, not the developers of the library) to understand the API so if we continue with this goal in mind, adding new DTOs and methods would make the most sense. Otherwise newbies will have to deal with complex "wrapper" classes for method arguments and return values. If someone comes and just want to place a limit order for a plain old "spot", they can easily find it by searching the methods and it is relatively easy then to understand what arguments are required and what is returned. As a compromise we could forgo adding new service classes but add more methods within the existing ones. By default the new methods will just throw a NotImplementedYetException
as it always has worked.
I think this would warrant a version bump to 4.5.0.
As far as extending Order
or CurrencyPair
or not I'd say yes if there is significant overlap of class members, but I'd say no if the members were completely or mostly orthogonal. Just take it by a case to case basis.
@mdvx If you think extending the Order class is not ideal, in which way do you think things could be improved?
Thank you @timmolter for your input.Indeed, you propose the simplest and easier solution.Much better, in my opinion. So, i will just add new DTOs and new methods to existing services.
Here are some different concepts for Order class...
Order -> OrderRequest The Order class does not allow for enough parameters, flags was added to extend the functionality, but things like GTD need fields. A MyExchangeOrder class can be created, but a lot of these are common across exchanges, and it would need to be applied to LimitOrder and StopOrder too. An interface for GTD might be a solution. Another possibility is to add a Map to the base Order object for extra the fields, then GTD could go in the map. GTD is just one such field, others might include LocateReqd flag, SettlCurrency, QuoteID, SettlType
Order -> OrderState of private Trade We are missing an error status here for rejected Orders, whereas this is not required when placing synchronous REST calls, asynchronous calls need a place to report this. REST are generally synchronous, but there is no reason an exchange would not reject an order further down its processing pipeline, maybe even after it is partially executed (risk limit hit after price moves)
Order -> public Trade This assumes a fully executed order, sometimes with maker and taker ids.
LimitOrder -> OrderBookEntry Problem here is the orderbook can combine many Orders for a single Entry (n<-1)
Order -> Position Well I thinks that is a lot to ask from one class
My tuppence.
With regards to extending the currency pair, I thought it might make sense as some derivatives will have a base/quote and a prompt month such as BTC.USD.PERPETUAL, even FTX's BTC-MOVE-1116 is really BTC.USD volatility plus a prompt, (i.e BTC.USD.MOVE.1116) so it has an additional symbol MOVE, where other derivatives such as TRUMP-2020 https://ftx.com/trade/TRUMP or SHIT-PERP don't have an underlying currency pair. So may be it makes more sense to have a new type instryment that has an underlying that can be anything, most cases it will be an CurrencyPair but could be something else like TRUMP or SHIT (shitcoin index)
On Wed, Mar 4, 2020 at 4:41 PM Tim Molter notifications@github.com wrote:
There are so many ways this could be done, which makes it hard to decide. We could add new DTOs, Services and methods. We could add new DTOs and methods to existing services. We could add new and/or extended DTOs to existing Services and existing methods.
I always tried to lean towards making it easy for the end user (a programmer using the library, not the developers of the library) to understand the API so if we continue with this goal in mind, adding new DTOs and methods would make the most sense. Otherwise newbies will have to deal with complex "wrapper" classes for method arguments and return values. If someone comes and just want to place a limit order for a plain old "spot", they can easily find it by searching the methods and it is relatively easy then to understand what arguments are required and what is returned. As a compromise we could forgo adding new service classes but add more methods within the existing ones. By default the new methods will just throw a NotImplementedYetException as it always has worked.
I think this would warrant a version bump to 4.5.0.
As far as extending Order or CurrencyPair or not I'd say yes if there is significant overlap of class members, but I'd say no if the members were completely or mostly orthogonal. Just take it by a case to case basis.
@mdvx https://github.com/mdvx If you think extending the Order class is not ideal, in which way do you think things could be improved?
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/knowm/XChange/issues/3171?email_source=notifications&email_token=ABBUO2RV7AFUFG6IMQ74DGTRF2AD5A5CNFSM4ILH5FDKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOENY2QSY#issuecomment-594651211, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABBUO2TGV5ZI2GTNCUJG45DRF2AD5ANCNFSM4ILH5FDA .
Here are the 4 near (+1 prior) month CME contracts BTC.G20 | Feb 2020 | 8825 | 8905 | 8430 | 8890 | -85 | -0.98% | 10:33 BTC.H20 | Mar 2020 | 8830 | 8910 | 8700 | 8775 | -25 | -0.28% | 17:28 BTC.J20 | Apr 2020 | 8940 | 8970 | 8795 | 8840 | -30 | -0.33% | 17:28 BTC.K20 | May 2020 | 9030 | 9070 | 8970 | 8970 | -20 | -0.22% | 17:28 BTC.M20 | Jun 2020 | 9150 | 9215 | 9035 | 9075 | -55 | -0.60% | 17:28
But we also have contracts on the spreads between two futures (J,K, are six week codes) BTC.G20:H20 | Feb 2020/Mar 2020 Spread | 41 | 80 | 41 | 77 | +35 | +51.47% | 10:33 BTC.H20:J20 | Mar 2020/Apr 2020 Spread | 85 | 90 | 80 | 80 | -12 | -17.65% | 17:28 BTC.H20:K20 | Mar 2020/May 2020 Spread | 201 | 201 | 190 | 190 | -12 | -5.97% | 17:28 BTC.H20:M20 | Mar 2020/Jun 2020 Spread | 314 | 314 | 300 | 300 | -25 | -7.96% | 17:28 BTC.J20:K20 | Apr 2020/May 2020 Spread | 103 | 111 | 101 | 108 | +2 | +1.82% | 17:28 BTC.J20:M20 | Apr 2020/Jun 2020 Spread | 199 | 252 | 199 | 225 | +55 | +27.64% | 17:28 BTC.K20:M20 | May 2020/Jun 2020 Spread | 103 | 105 | 103 | 105 | -14 | -13.59% | 17:28
here is one with 3 currencies Litecoin (BITCOIN:LTCUSD)
https://quotes.ino.com/exchanges/category.html?c=cryptocurrencies
If we think to the definition of a Future contract, it is an obligation to Buy/Sell and asset at a future date (BTC). Sometimes these contracts are settled in the quoted fiat (USD), other times these contracts are settled for 'physical' delivery, which would mean in actual BTC. In which case, the USD part is just a reference, and this very same contract could be quoted in JPY, EUR, or even ETH.
As such a futures contract is not a currency pair, but depends on a single underlying asset.
so based on above, how do people feel about Introduce a new types of instruments, say two concert classes of FuturesContract and OptionsContract (options possibly extends futures contract as it will have a lot of the same attributes, plus type, strike etc). Futures contract which will have many attributes one of which is underlying, which is an instrument which could be a CurrencyPair or any other instrument we support, for example an option could have a underlier of a FuturesContact.
Instrument (Abstract Class)
|___ CurrencyPair
|___ FuturesContract
|__OptionsContract
OptionsContract should not extend FuturesContract as options can be on the underlying asset as well as the Future. Simple Vanilla options will have a StrikePrice and an ExpiryDate, however they get a lot more complex, some with multiple ExpiryDates (Asian options), and many other types of derivations.
@mdvx also, according to what you said:
but this would allow an OptionsContract on an OptionsContract - could this be a thing in the real world? if not we can always ad some validation in the DTO
coolio, so just this then, where the Futures and Options contract can have an underlier of type Instrument.
Instrument (Abstract Class)
|___ CurrencyPair
|___ FuturesContract
|__OptionsContract
coolio, so just this then, where the Futures and Options contract can have an underlier of type Instrument.
Instrument (Abstract Class) |___ CurrencyPair |___ FuturesContract |___ OptionsContract
This looks like the right structure, to me
There was a question about Options on Options I saw last night. These things do exists, as compound options, however they are the exceptional, and as such I would not be inclined to model them here.
Don't think it matters, as both option contracts and futures contracts will have an underlier of instrument which would allow for options on currencypairs, futures as well as on options.
On Fri, 6 Mar 2020, 15:34 Marc Deveaux, notifications@github.com wrote:
There was a question about Options on Options I saw last night. These things do exists, as compound options, however they are the exceptional, and as such I would not be inclined to model them here.
https://www.investopedia.com/terms/c/compoundoption.asp
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/knowm/XChange/issues/3171?email_source=notifications&email_token=ABBUO2QZMED6FCPTYTVCSTTRGEJW3A5CNFSM4ILH5FDKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEOBYNJI#issuecomment-595822245, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABBUO2X6DN2HL6OL2JOYTMLRGEJW3ANCNFSM4ILH5FDA .
@mdvx @walec51 anyone started looking at this on a branch? Happy to collaborate or I can start a branch early next week.
this https://github.com/knowm/XChange/pull/3461 would break the entire API so I doubt it will get merged
few exchanges have futures / options and therefore we talked to make either separate methods or even separate interfaces for them
wont the orders for options / futures have different fields and behaviors then the current LimitOrder
, MarketOrder
and SpotOrder
?
if so then there is no sense in pretending that they can accept any thing other then a CurrencyPair
and I would suggest to change the Instument
class hierarchy to
Instrument (Abstract Class)
|___ CurrencyPair
|___ Derivative
|___ FuturesContract
|___ OptionsContract
and instead of breaking the current API to add methods (or even separate interfaces) that can handle Derivative
s
@makarid in this PR https://github.com/knowm/XChange/pull/3439 suggested that the fields for and order on a derivative must by totally different
if he is wrong and orders for derivatives can be handled by our existing order type from the core API (after introducing the Instrument class to them) then I'm in favor of breaking the core API to introduce the Instrument class - however this will mean an entirely new version of our library and force all our users to refactor their code
so the crucial question we have to answer before we choose a direction is:
can our current core API Order
(LimitOrder
, SpotOrder
, MarketOrder
), Trade
and Ticker
classes be made to work for derivatives if we only replace CurrencyPair
with Instument
in them?
aren't there any specific fields that a derivative would have to have in an Order
, Trade
or Ticker
class?
Hi adam,
Thanks so much for all your input, nothing like a PR to get things flowing!
From getting market data (MarketdataService
) and placing orders (TradeService
), I am pretty sure the exchange api's are very simliar, just the symbol changes to incorporate the pair, expiry and additionally strike and put/call for options. Hence my thinking was to reuse them but support more types of instruments, whos concrete class contain enough information to create the required symbol for the exchange's api.
I think @makarid point is that we would require additional API's to handle other things such as position, (PositionService
), but are these conceptually any different from the additional services we already support such as AcountService
?
DEPTH
Karken
https://futures.kraken.com/derivatives/api/v3/orderbook?symbol=fi_xbtusd_180615
https://api.kraken.com/0/public/Depth?pair=XBTCHF
OKEX
https://www.okex.com/api/spot/v3/instruments/BTC-USDT/book?size=5&depth=0.2
https://www.okex.com/api/futures/v3/instruments/BTC-USD-180309/book?size=50
Binance
https://api.binance.com/api/v3/depth?symbol=ETHBTC
https://fapi.binance.com/fapi/v1/depth?symbol=ETHUSDT
Orders
Binanace
https://api.binance.com/api/v3/order{"symbol":"ETHBTC","side":"SELL","type":"MARKET","quantity":"0.123"}
https://fapi.binance.com/fapi/fapi/v1/order{"symbol":"BTCUSDT","side":"BUY","type":"MARKET","quantity":"1"}
OKEX
https://www.okex.com/api/spot/v3/orders{"type": "limit", "side": "buy", "instrument_id": "BTC-USDT", "size":"0.001", "client_oid": "oktspot79", "price": "4638.51", "notional": "", "order_type": "3"}
https://www.okex.com/api/futures/v3/order{"client_oid": "y12233456","order_type":"1","instrument_id":"BTC-USD-180213","type":"1","price":"432.11","size":"2","match_price":"0"}
Karken
https://api.kraken.com/0/private/AddOrder{"pair":"XBTCHF", "type":"buy","ordertype":"market","volume":"0.123"}
https://futures.kraken.com/derivatives/api/v3/sendorder{"orderType":"lmt","symbol":"fi_xbtusd_180615","side":"buy","size":"1","limitPrice":"2321.2"}
Derbit
https://www.deribit.com/api/v2/public/get_order_book?instrument_name=BTC-22MAR20-5375-C
https://www.deribit.com/api/v2/private/buy{"instrument_name":"BTC-22MAR20-5375-P", "amount":"buy","type":"market","amount":"1"}
whats the difference between a position and an order? why would we have both for futures?
in PR #3439 you have
Position extends Order
if we would want to handle futures your our existing API (just by changing currencyPair to instrument) then having a position class like this makes no sense at alll
Conceptually it is like an amount, basically a running total of your orders.
Buy +10 Sell -5 Buy +4
My position would be 9. When trading spot you are buying and selling currencies, so your position is your balance. For derivatives you are buying and selling contracts. So you have two things, your position (basically the running total of contracts your have bought and sold) and your balance, the actual amount of currencies in your account.
One thing for derivatives is we need to specify if the order is opening or closing but this is already supported on the order dto. In the above example, if all orders were opening you would have two positions of buy/long +14 and sell/short of -5, rather than a single position of buy/long +9.
A position service is not critical in my.mind, firstly not offered by all exchanges, secondly the client application could keep track of the position based on the fills from the orders, just like the client application needs to maintain order state to know which orders are working and closed.
On Sun, 22 Mar 2020, 10:36 Adam Walczak, notifications@github.com wrote:
whats the difference between a position and an order? why would we have both for futures?
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/knowm/XChange/issues/3171#issuecomment-602178035, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABBUO2XIMLMPUQQBVI2ODPLRIXS3BANCNFSM4ILH5FDA .
if it should represent a collection of orders then I understand that this makes no sense:
Position extends Order
?
Correct, position should not extend order, the only real common items are the instrument and quantity. Position is the aggregation/running total of the fills not a collection of orders. Might place a buy order of 100, only get filled 10, then only the fill of 10 would be added to my position. Also a fill would be removed from a position once it has been closed, i.e its quantity has been offset by a closing fill of the same but opposite quantity. Also the exchanges don't tell you which fills make up.a position, so unless xchange is planning to keep all the fill state, I would suggest just having position as its own object.
You have a position only when you use leverage. That way your balance doesn't change(you don't sell bitcoin and get euro for example.). A position is opened with exchange's money, using your balance as collateral. An order is when you exchange X currency with Y currency which will change you balance. So in order to know how much money you have, when you open a position, you need to track open positions value.
@douggie if your thesis that orders for derivatives can be represented in the same way as orders for normal currency pairs then I'm in favor of your solution
however they way you prepared your pull request is not the way to go - reviewing 400 files is impossible and no one would upgrade to the new version because it would break every existing code that uses XChange
I think we should slice out one aspect at a time out of it and try to make the API backwards compatible
lets only start with the Order
and just do:
public abstract class Order implements Serializable {
// remove:
// private final CurrencyPair currencyPair;
// and replace with
private final Instrument instrument;
// in all constructors in Order replace 'CurrencyPair currencyPair' with 'Instrument instrument`
// provide these methods
@Deprecated
public CurrencyPair getCurrencyPair() {
if (!(instrument instanceof CurrencyPair)) {
throw new IllegalStateException("The instrument of this order is not a currency pair: " + instrument);
}
return (CurrencyPair) instrument;
}
public Instrument getInstrument() {
return instrument;
}
// in the builder:
@Deprecated
public Builder currencyPair(CurrencyPair currencyPair) {
this.instrument= currencyPair;
return this;
}
public Builder instrument(Instrument instrument) {
this.instrument= instrument;
return this;
}
}
if you apply my suggestion above correctly then you will not have to modify all the exchange implementations in this project
and you will be able to add support for placing future orders in TradeService
place*Order
in the few exchanges that support it
lets do this for a start, this should be a PR of no more then a dozen files
@makarid
You have a position only when you use leverage
is a position something specific to derivatives? or can you have a position on a normal currency pair like BTC/USD?
Currently the central key point of the XChange API is the CurrencyPair, which is passed as parameter for almost all methods. This works fine so far for spot markets. But if we want to support futures markets the CurrencyPair is not enough, for example see Deribit Instruments, there we have three "products" with the same pair BTC/USD. To support this we would need to replace the CurrencyPair by
Product
class, which could look like this:To stay with the deribit example, we would have three instances for products:
So like already said, inside the interface methods we could replace the CurrencyPair by Product (or Instrument or what ever name). The old methods could actually stay for a while and be marked as deprecated. This way we would be still compatible with older versions.
if the majority thinks it could make sense, i will create a PR