Get Marketdata

[sf202020sf]

Hi, i tried to implement a stream with the onUpdateSummaryState Method a stream in order to recieve all updates from all coins - it seams that bittrex blocks the data stream or reduce the speed - to you know how to get all actual ticks from all coins? I would like to build 1 Min candle from all coins - do you know how i should implement this with your lib?

Thx!

[CCob]

Affraid there is not a lot you can do about that. The more markets you subscribe to the less realtime it gets. Unless you have a corporate account with Bittrex. Your only other option is subscribing to different markets from different IP addresses.

[CCob]

You can use the REST API, but this is not realtime either

[sf202020sf]

thx a lot for your replay: I tested currently the following method from your example file: bittrexExchange.onUpdateSummaryState(exchangeSummaryState -> { there I can get 100 Updates on every 5 second so it seams that bittrex does not throttle the throughput with this method - it is very constant.

But my problem is now how to proceed with these data: when I call a new method in order to start my calculation (this would take longer than 5 seconds) I get a backlog in the data stream - have you tried something like that? Do I need a parallel thread for that?

[CCob]

If your callbacks for onUpdateSummaryState and onUpdateExchangeState have expensive CPU operations then yes you should handle your calculations in a separate thread to prevent missing/dropping updates from the exchange

[CCob]

I'll close this now, since this is not an issue with bittrex4j. Thanks

[sf202020sf]

ok i tested now a couple of things but it is not possible to recieve realtime data: getTicks --> delivery you a couple of data but is 3! min delayed - so no chance to build here some candles

onUpdateSummaryState --> is also delayed about 3 seconds so every candle will be wrong and not compareable

So there is no chance to get actual data from bittrex

[CCob]

If you want to build candle data, getTicks nor onUpdateSummaryState are the right approaches. The REST API's are cached and your observation of around 2-3 mins is correct. If you want to rely on hourly candles or more, the REST API is fine, but if you building minute by minute candles you can forget the REST API. What you should be doing is using subscribeToExchangeDeltas, queryExchangeState and onUpdateExchangeState and track the order fills sequentially.

[sf202020sf]

but subscribeToExchangeDeltas needs a pair as input - so i can not calculate it for the complete market...

[CCob]

Hmm, not sure what your use case is. That's exactly what you build candles from, a single market pair. Now if you are interested in multiple markets at the same time then you can all subscribeToExchangeDeltas multiple times. But be warned the more pairs you subscribe to the slower it can get unless you have a corporate style feed from Bittrex.

[Georim]

May be it can help sf202020sf,

1. Obtain list of pairs by rest request (https://bittrex.com/api/v1.1/public/getmarkets) or by BittrexExchange.getMarkets() which do, as I understand, the same.
2. Make several connections which is subscribed by subscribeToExchangeDeltas to not more than 60 symbols.
3. Collect obtained data as you need.

[CCob]

@Georim - yup, that is certainly the process you would follow.

[Georim]

Ccob, thank you for your library. It is very helpful. One more thing, it very useful to add the public method which can close connection at all without inner re-connection watcher. I have done it, and my "watch dog" make sophisticated reconnect when BittrexExchange connection is ok, but amount of socket error is not admissible.

[sf202020sf]

ok thank for your replay: I am currently doing the following: bittrexExchange.subscribeToExchangeDeltas("BTC-ETH", null);

and i listen to an update on the following method:

bittrexExchange.onUpdateSummaryState(exchangeSummaryState -> {

but i am not sure if this is correct --> which method is the right on in order to listen to updates to ExchangeDeltas? the onUpdateExchangeState ?

You wrote that I should listen to all fills an track them and calc manually the candle --> what is the logic behind? On every will I take the price as an tick and put them to the candle (and calc. the Low, High etc.)?

[CCob]

Check this out for background on how you process orders. It's for the NodeJS library, but the principle is the same of bittrex4j

https://github.com/n0mad01/node.bittrex.api/files/1104308/WebSocketAPI_MarketTracking.docx

[CCob]

@Georim, there is a close method and it's AutoClosable too if you prefer. Does that not work for you?

[Georim]

No, if you close BitrexExchange by that close method, inner connection watcher will reconnect you back after 5 seconds. See reconnectTimer.schedule(new ReconnectTimerTask(),5000); So, close method should disable reconnectTimer for example.

[CCob]

Ah OK, understood. So it just needs to be updated so that if close is called the reconnectTimer is disabled. Do you mind creating a issue for this. I can sort that out for the next release. Thanks.

[sf202020sf]

ok thanks for your reply - I checked this out: I register currently like the following

` bittrexExchange.connectToWebSocket(() -> {

            bittrexExchange.subscribeToExchangeDeltas("BTC-ETH", null);
            bittrexExchange.subscribeToExchangeDeltas("BTC-BCH", null);`

and recieve the updates but the following method

bittrexExchange.onUpdateExchangeState(updateExchangeState -> {

             String marketname = updateExchangeState.getMarketName();

             Fill [] fill = updateExchangeState.getFills();    

             for (int j = 0; j < fill.length; j++) {
                 print("Fillprice Update: " + fill[j].getPrice() 
                         + " Ordertype: " + fill[j].getOrderType() 
                         + " ID: "+fill[j].getId()
                         + " Total "+ fill[j].getTotal()
                         + " Name "+ marketname
                 );
             }
        });

works fine now so this streams looks good and i exact the same like in tradingview :-)

but what for do I need the method: queryExchangeState

is it in order to get data manually (buy request?) - the nounce value is an increasing number which shows me if i miss something or? but there exists no "listener" for that or?

so for my use case (getting data for couple of market data and build 1min candle) I don't need the query or?

[CCob]

listen to the callback to subscibeToExchangeDeltas, once you get a success, it means you are registered for the updates. In the callback you can then call queryExchangeState, which gives you a full (max 500 orders) order book and a starting Nonce value. Remember the starting nonce and process the fills to build your candles, in the mean time you will get updates to the order book via onUpdateExchangeState with incrementing nonce values. If the nonce value is not sequential from your nonce you got via queryExchangeState then it means you have lost sync and need to start over.

[sf202020sf]

Thx a lot I implemented that - works fine now!!

One more question: I checked my candles against tradingview and I have noticed small differences: Somethings the exchange send an update with the same milliseconds but with different prices (don't know how this can happen but I can see this in my stream) - tradingview is ignoring the second the one and takes just the first one - in my logic I will take the last value - this causes some differneces do you know what is the correct way?

Second question: will be the nounce resettet? currently the nounce at BTC-ETH is at 1823 this is very low - will be the value resettet every day? if yes I have to consider that or?

[CCob]

Without seeing your candle building logic it's difficult for me to see why there could be differences.

You shouldn't really be concerned with what number the nonce is, just that it's going up in a sequential manner from the initial queryExchangeState to the subsequent onUpdateExchangeState messages. They can and do change, specifically if the websocket has disconnected for some reason and you end up connecting to a different Bittrex API server as part of their cluster.

[sf202020sf]

I think it is not based on my candle caluclation - I think it is a rule in tradingview - here an example: you get the following data from bittrex: 12:01:58.023 Price: 0,7 12.01:58.023 Price 0,8

Lets say this is the last update for the candle 12:01.000 - 12:01:999 from the exchange. In my calculation the close price in this candle is 0,8 but on tradingview it is 0,7 - thats really strange

I will test this once more in order to see this pattern several times and let you know

[sf202020sf]

hi again - I have seen that I have an issue with the close and open price - what is the real pattern behind the open price? a.) is the open price the last close? b.) is the open price the first update in the stream?

another question: currently I consider all data in the method onUpdateExchangeState - Buy and Sells - is this correct? or do I have to consider just one?

[sf202020sf]

Here another strange behaviour (like I described abouth):

I get the following data:

27-06 18:46:18 FINE Added Streamdata: Time : 18:46:20.00193 Price: 0.07048305 Name: BTC-ETH Nounce:25504 Ordertype: SELL

27-06 18:46:47 FINE Added Streamdata: Time : 18:46:49.00370 Price: 0.07081999 Name: BTC-ETH Nounce:25531 Ordertype: BUY

27-06 18:46:47 FINE Added Streamdata: Time : 18:46:49.00370 Price: 0.07063832 Name: BTC-ETH Nounce:25531 Ordertype: BUY

and I build this candle:

27-06 18:47:00 INFO BTC-ETH: From: 18:46:00.000 To: 18:46:59.999 O_Price: 0.07048305 H_Price: 0.07081999 L_Price: 0.07048305 C_Price: 0.07063832

so my close is 0.07063832 - which I took from the last data which I got from the Exchange - but the close of the candle 18:46 in coin ETHBTC is 0.0708199 (which is the previous one) - thats really strange and I don't know which pattern is behind.

This i may code in the method bittrexExchange.onUpdateExchangeState((UpdateExchangeState updateExchangeState) -> {

`Fill[] fill = updateExchangeState.getFills();

            if (fill.length > 0) {
                for (int j = 0; j < fill.length; j++) {
                    long current_candle_start = System.currentTimeMillis() - (System.currentTimeMillis() % interval);
                    Fill singlefill = fill[j];

                    String stream_name = marketname;
                    long stream_time = singlefill.getTimeStamp().toInstant().toEpochMilli();
                    String stream_price = String.valueOf(singlefill.getPrice());

                    Candlestick calculatedcandle = this.calcCandle(current_candle_start, stream_time, stream_name, stream_price);

                    bittrex.log(Level.FINE, "Added Streamdata:"
                            + " Time : " + sdateformat.format(stream_time)
                            + " Price: " + stream_price
                            + " Name: " + stream_name
                            + " Nounce:" + actual_nounce
                            + " Ordertype: "+ singlefill.getOrderType()
                    );
                }
            }`

[sf202020sf]

Hi again I checked this once more and it seems there is difference to tradingview - i get the following data

28-06 19:54:41 FINE Added Streamdata: Time : 19:54:40.00487 Price: 0.07151 Name: BTC-ETH 28-06 19:54:41 FINE Added Streamdata: Time : 19:54:40.00487 Price: 0.07150999 Name: BTC-ETH 28-06 19:54:41 FINE Added Streamdata: Time : 19:54:40.00487 Price: 0.07150999 Name: BTC-ETH 28-06 19:54:41 FINE Added Streamdata: Time : 19:54:40.00487 Price: 0.07131646 Name: BTC-ETH

Tradingview ignores the last 3 updates and shows a price at 0.07151 - but in my view it should be 0.7131 - any ideas on this?

[CCob]

I need to see your logic on where you get the price from

[sf202020sf]

`bittrexExchange.onUpdateExchangeState((UpdateExchangeState updateExchangeState) -> { String marketname = updateExchangeState.getMarketName();
long actual_nounce = updateExchangeState.getNounce();
Fill[] fill = updateExchangeState.getFills();

            if (fill.length > 0) {
                for (int j = 0; j < fill.length; j++) {
                    long current_candle_start = System.currentTimeMillis() - (System.currentTimeMillis() % interval);
                    Fill singlefill = fill[j];

                    String stream_name = marketname;
                    long stream_time = singlefill.getTimeStamp().toInstant().toEpochMilli();
                    String stream_price = String.valueOf(singlefill.getPrice());

bittrex.log(Level.FINE, "Added Streamdata:"

• " Time : " + sdateformat.format(stream_time)
• " Price: " + stream_price
• " Name: " + stream_name
• " Nounce:" + actual_nounce
• " Ordertype: "+ singlefill.getOrderType()
• " Filltype: "+singlefill.getFillType() );`

[sf202020sf]

here an other example for this behaviour (ETH/BTC exchange time +2h):

28-06 22:47:03 FINE Added Streamdata: Time : 22:47:02.00000267 Price: 0.07155973 Name: BTC-ETH Nounce:35404 Ordertype: BUY Filltype: null 28-06 22:47:05 FINE Added Streamdata: Time : 22:47:05.00000080 Price: 0.07155974 Name: BTC-ETH Nounce:35406 Ordertype: BUY Filltype: null 28-06 22:47:10 FINE Added Streamdata: Time : 22:47:10.00000110 Price: 0.07155976 Name: BTC-ETH Nounce:35411 Ordertype: BUY Filltype: null 28-06 22:47:11 FINE Added Streamdata: Time : 22:47:11.00000110 Price: 0.07155976 Name: BTC-ETH Nounce:35412 Ordertype: BUY Filltype: null 28-06 22:47:19 FINE Added Streamdata: Time : 22:47:18.00000830 Price: 0.07155973 Name: BTC-ETH Nounce:35419 Ordertype: SELL Filltype: null 28-06 22:47:19 FINE Added Streamdata: Time : 22:47:18.00000830 Price: 0.07155974 Name: BTC-ETH Nounce:35419 Ordertype: SELL Filltype: null 28-06 22:47:22 FINE Added Streamdata: Time : 22:47:21.00000783 Price: 0.07155976 Name: BTC-ETH Nounce:35421 Ordertype: BUY Filltype: null 28-06 22:47:22 FINE Added Streamdata: Time : 22:47:21.00000703 Price: 0.07141003 Name: BTC-ETH Nounce:35421 Ordertype: BUY Filltype: null

this is to complete stream of the 1 min candle of 22:47:000 - 22:47.999

Tradingview shows a candle with the following value: Open -->0.07155973 (correct) High --> 0.07155976 (correct) Low --> 0.07141003 (correct) Close -->0.07155976 (NOT CORRECT) because there was one additional tick with an price of 0.07141003 so the close must be 0.07141003

the strange thing is that tradingview considers the 0.07141003 as low but not as close!

[CCob]

As long as you are not using the local clock to determine where the fills go into which candle you should be fine. Generally you can't close the incomplete candle until you get the first fill that goes beyond the candle you are currently building. You can't rely on your local clock for this since the fill notifications could be delayed by a few seconds (or more)

[sf202020sf]

thx I will check that - there is an other issue which i found - I have to consider the datetime of the ticks and sort them - in the last example the last tick came late and was in reality not the close because the milliseconds was smaller than the previous one

[CCob]

I can't say I've seen the fills coming in out of order before mind, but in that case you simply wound't adjust the close price of the candle you are building since the last trade timestamp within your candle would be newer. You would just update the volume/high/low if needed.

Now you have a different problem if it comes in out of order and the out of order entry is actually a close on the previous candle that you have now deemed complete.

[sf202020sf]

ok but thats a real issue --> in my case the ticks don't come in the right order --> thats fine for me I can sort them --> but what I currently see is that the ticks are coming with the same milliseconds. And in that case the order can not be done--> I don't know which one is the open and which on is the close --> I currently take the milliseconds from the stream with the following code:

`Fill[] fill = updateExchangeState.getFills(); //the content of the update

            if (fill.length > 0) {
                for (int j = 0; j < fill.length; j++) {

                    Fill singlefill = fill[j];

                    String stream_name = marketname;
                    long stream_time = singlefill.getTimeStamp().toInstant().toEpochMilli();`

[CCob]

If they are at the exact same time one cant be the close and the other cant be the open since they are the same time. Only one of them will be and personally I would take the order they appear

[sf202020sf]

ok thx - I have checked that against tradingview --> tradingview takes the first one (in case the milliseconds are equal) --> the second and the third a completly ignored (for open and close) --> I haven't checked yet for Low and High but it seems to be the same pattern