Closed chiconguyen2601 closed 6 years ago
aloysius-pgast
commented
6 years ago Hi What do you mean with « after a while » ? (seconds, minutes, hours) ? Do you still receive orderbook updates in the meantime ? Does it happen only with those pairs ?
chiconguyen2601
commented
6 years ago Hi
Thank you so much for your reply.
It's usually one hour or two hour. I just subscribe on TRADES event for all Market pairs, but after 1 hour or 2 hours, I don't receive any data in "on('trades')" event.
I think it have been unsubscribed from market (bittrex). But I how to immediately detect this problem once it happen.
aloysius-pgast
commented
6 years ago Could you also try to subscribe to 'orderBookUpdate' and 'terminated' events and let me know if you still receive 'orderBookUpdate' when you stop receiving 'trades' ?
chiconguyen2601
commented
6 years ago I also subscribed 'terminated' event. But also receive nothing in this event. I see the log and saw that the connection is still kept.
Okay, let me also subscribe to 'orderBookUpdate' and test it now.
aloysius-pgast
commented
6 years ago Great. Please also subscribe to 'disconnected', 'connectionError' & 'connected' so that we have a better understanding of the underlying problem. Thanks
chiconguyen2601
commented
6 years ago Hi Just happen same problem in this test time. I subscribed to 'disconnected', 'connectionError', 'connected', 'trades' and 'orderBookUpdate'.
I receive nothing in these events: 'disconnected', 'connectionError', 'trades' and 'orderBookUpdate'.
I run 4 instances: one instance subscribe all market pairs, one instance subscribe all BTC-market pairs, one instance subscribe all ETH-market pairs, and remain one subscribe all USDT-market pairs.
Could you help check why?
aloysius-pgast
commented
6 years ago I have instances running for days without problems. Which version are you using ?
chiconguyen2601
commented
6 years ago I'm using version 1.0.7. Could you share version you're using? This is my source code which using your library. Is it correct? `
const SignalRClient = require('bittrex-signalr-client');
const logger = require('winston');
//*****************MQTT****************//
var asyncMQTT = require("async-mqtt");
var asyncMQTTClient = asyncMQTT.connect("tcp://localhost:1883");
var mqttConnected = false;
// MQTT
asyncMQTTClient.on("connect", function() {
console.log("Connected MQTT Broker");
mqttConnected = true;
});
let client = new SignalRClient({
userAgent:'CEG 1.0',
pingTimeout:10000,
logger:logger,
reconnectAfterUnsubscribingFromMarkets:{reconnect:false}
});
client.on('connected', function(data){
console.log("connected");
});
client.on('disconnected', function(){
console.log("disconnected");
// nothing to do, reconnection will be automatic
});
// no more retry, we need to reconnect
client.on('terminated', function(){
console.log("terminated");
client.reconnect(false);
});
// client.on('ticker', function(evt){
// });
// client.on('orderBook', function(evt){
// });
client.on('orderBookUpdate', function(evt){
console.log("orderBook" + evt);
});
var data_length = 0;
client.on('trades', function(data){
if (data.data != null) {
if (data.data.length == 0) {
var now = new Date();
console.log(now + ": Data Empty" )
}
// console.log(data);
data.data.forEach(element => {
if (mqttConnected) {
var date = new Date();
var now = parseFloat(date.getTime()/1000.0);
var spliter = ","
// pair|quantity|rate|price|orderType|timestamp|created
var mqttMessage = data.pair + spliter + element.quantity + spliter + element.rate
+ spliter + element.price + spliter + element.orderType
+ spliter + parseInt(element.timestamp)
console.log(mqttMessage);
asyncMQTTClient.publish('foo', JSON.stringify(mqttMessage));
}
});
} else {
var now = new Date();
console.log(now + ": Data NULL" )
}
});
client.connect();
client.subscribeToMarkets(["BTC-1ST",
"BTC-2GIVE",
"BTC-ABY",
"BTC-ADA",
"BTC-ADT",
"BTC-ADX",
"BTC-AEON",
"BTC-AGRS",
"BTC-AMP",
"BTC-ANT",
"BTC-APX",
"BTC-ARDR",
"BTC-ARK",
"BTC-AUR",
"BTC-BAT",
"BTC-BAY",
"BTC-BCC",
"BTC-BCY",
"BTC-BITB",
"BTC-BLITZ",
"BTC-BLK",
"BTC-BLOCK",
"BTC-BNT",
"BTC-BRK",
"BTC-BRX",
"BTC-BSD",
"BTC-BTCD",
"BTC-BTG",
"BTC-BURST",
"BTC-BYC",
"BTC-CANN",
"BTC-CFI",
"BTC-CLAM",
"BTC-CLOAK",
"BTC-CLUB",
"BTC-COVAL",
"BTC-CPC",
"BTC-CRB",
"BTC-CRW",
"BTC-CURE",
"BTC-CVC",
"BTC-DASH",
"BTC-DCR",
"BTC-DCT",
"BTC-DGB",
"BTC-DMD",
"BTC-DNT",
"BTC-DOGE",
"BTC-DOPE",
"BTC-DTB",
"BTC-DYN",
"BTC-EBST",
"BTC-EDG",
"BTC-EFL",
"BTC-EGC",
"BTC-EMC",
"BTC-EMC2",
"BTC-ENG",
"BTC-ENRG",
"BTC-ERC",
"BTC-ETC",
"BTC-ETH",
"BTC-EXCL",
"BTC-EXP",
"BTC-FAIR",
"BTC-FCT",
"BTC-FLDC",
"BTC-FLO",
"BTC-FTC",
"BTC-FUN",
"BTC-GAM",
"BTC-GAME",
"BTC-GBG",
"BTC-GBYTE",
"BTC-GCR",
"BTC-GEO",
"BTC-GLD",
"BTC-GNO",
"BTC-GNT",
"BTC-GOLOS",
"BTC-GRC",
"BTC-GRS",
"BTC-GUP",
"BTC-HMQ",
"BTC-INCNT",
"BTC-INFX",
"BTC-IOC",
"BTC-ION",
"BTC-IOP",
"BTC-KMD",
"BTC-KORE",
"BTC-LBC",
"BTC-LGD",
"BTC-LMC",
"BTC-LSK",
"BTC-LTC",
"BTC-LUN",
"BTC-MAID",
"BTC-MANA",
"BTC-MCO",
"BTC-MEME",
"BTC-MER",
"BTC-MLN",
"BTC-MONA",
"BTC-MUE",
"BTC-MUSIC",
"BTC-MYST",
"BTC-NAV",
"BTC-NBT",
"BTC-NEO",
"BTC-NEOS",
"BTC-NLG",
"BTC-NMR",
"BTC-NXC",
"BTC-NXS",
"BTC-NXT",
"BTC-OK",
"BTC-OMG",
"BTC-OMNI",
"BTC-PART",
"BTC-PAY",
"BTC-PDC",
"BTC-PINK",
"BTC-PIVX",
"BTC-PKB",
"BTC-POT",
"BTC-POWR",
"BTC-PPC",
"BTC-PTC",
"BTC-PTOY",
"BTC-QRL",
"BTC-QTUM",
"BTC-QWARK",
"BTC-RADS",
"BTC-RBY",
"BTC-RCN",
"BTC-RDD",
"BTC-REP",
"BTC-RISE",
"BTC-RLC",
"BTC-SALT",
"BTC-SBD",
"BTC-SC",
"BTC-SEQ",
"BTC-SHIFT",
"BTC-SIB",
"BTC-SLR",
"BTC-SLS",
"BTC-SNRG",
"BTC-SNT",
"BTC-SPHR",
"BTC-SPR",
"BTC-START",
"BTC-STEEM",
"BTC-STORJ",
"BTC-STRAT",
"BTC-SWIFT",
"BTC-SWT",
"BTC-SYNX",
"BTC-SYS",
"BTC-THC",
"BTC-TIX",
"BTC-TKS",
"BTC-TRST",
"BTC-TRUST",
"BTC-TX",
"BTC-UBQ",
"BTC-UKG",
"BTC-UNB",
"BTC-VIA",
"BTC-VIB",
"BTC-VOX",
"BTC-VRC",
"BTC-VRM",
"BTC-VTC",
"BTC-VTR",
"BTC-WAVES",
"BTC-WINGS",
"BTC-XCP",
"BTC-XDN",
"BTC-XEL",
"BTC-XEM",
"BTC-XLM",
"BTC-XMG",
"BTC-XMR",
"BTC-XMY",
"BTC-XRP",
"BTC-XST",
"BTC-XVC",
"BTC-XVG",
"BTC-XWC",
"BTC-XZC",
"BTC-ZCL",
"BTC-ZEC",
"BTC-ZEN",
"ETH-1ST",
"ETH-ADA",
"ETH-ADT",
"ETH-ADX",
"ETH-ANT",
"ETH-BAT",
"ETH-BCC",
"ETH-BNT",
"ETH-BTG",
"ETH-CFI",
"ETH-CRB",
"ETH-CVC",
"ETH-DASH",
"ETH-DNT",
"ETH-ENG",
"ETH-ETC",
"ETH-FCT",
"ETH-FUN",
"ETH-GNO",
"ETH-GNT",
"ETH-GUP",
"ETH-HMQ",
"ETH-LGD",
"ETH-LTC",
"ETH-LUN",
"ETH-MANA",
"ETH-MCO",
"ETH-MYST",
"ETH-NEO",
"ETH-NMR",
"ETH-OMG",
"ETH-PAY",
"ETH-POWR",
"ETH-PTOY",
"ETH-QRL",
"ETH-QTUM",
"ETH-RCN",
"ETH-REP",
"ETH-RLC",
"ETH-SALT",
"ETH-SC",
"ETH-SNT",
"ETH-STORJ",
"ETH-STRAT",
"ETH-TIX",
"ETH-TRST",
"ETH-UKG",
"ETH-VIB",
"ETH-WAVES",
"ETH-WINGS",
"ETH-XEM",
"ETH-XLM",
"ETH-XMR",
"ETH-XRP",
"ETH-ZEC",
"USDT-ADA",
"USDT-BCC",
"USDT-BTC",
"USDT-BTG",
"USDT-DASH",
"USDT-ETC",
"USDT-ETH",
"USDT-LTC",
"USDT-NEO",
"USDT-NXT",
"USDT-OMG",
"USDT-XMR",
"USDT-XRP",
"USDT-XVG",
"USDT-ZEC"]); ` With above code, you receive nothing at all or you receive events at first and then nothing ?
chiconguyen2601
commented
6 years ago Hi, I receive data on events 'trades' and 'orderBookUpdate' for one hour, then nothing come.
chiconguyen2601
commented
6 years ago Hi, How many instances are you running for days? I have run 4 instances from this afternoon (5 hours ago, as I said above) and the results as following:
How to detect when it is unsubscribed from market (bittrex)? I've written logs, but can not know why.
Thank you very much.
aloysius-pgast
commented
6 years ago I have one instance currently running since 21/01 but it is only subscribed to one pair. I have another instance which has been running for =~ 1 week but it is configured to reconnect everytime I unsubscribe from one market. And https://mpe-demo.crazyme.net/ has been running for =~ 20h also. 30 min ago, I started another instance which subscribe to all pairs as in https://github.com/aloysius-pgast/bittrex-signalr-client/issues/4#issuecomment-363739965. Code can be see here https://pastebin.com/uZ6BSnxb (it is configured to log orderBook, orderBookUpdate & trades only every 10 min)
chiconguyen2601
commented
6 years ago Great. Thank you very much.
Please kindly keep track of this test case. I will borrow your code to run on my machine for checking the logs also.
Tks so much again.
aloysius-pgast
commented
6 years ago For now, after 5h, I'm still receiving updates
chiconguyen2601
commented
6 years ago Yes, your code running on my machine is still receiving updates for now (after 9h). Strange, what's different on my source code and your source code?
chiconguyen2601
commented
6 years ago Hi, I just detect one problem as following:
You can see the logs in this image (https://drive.google.com/file/d/1nwXB5H5iK5H8L6beg7A85HR96_6cyKuu/view) for more detail. Do you know why and fix it?
Thank you very much.
chiconguyen2601
commented
6 years ago After reconnected, it has just received updates for 10 minutes, after that nothing come till now (4 hours already). You can check the logs in this image (https://drive.google.com/file/d/1M1v5oduYhRFtognxW3kDcTdcnK9w3Vni/view) for more detail.
chiconguyen2601
commented
6 years ago How about your logs now?
aloysius-pgast
commented
6 years ago Hi. I also had similar problem around 5:30 UTC. I just pushed a fix to implement a watchdog which will detect timeout on data an automatically reconnect. Can you pull the repository and give it a try ? You can use https://pastebin.com/fF9WtiEp as sample code (put the file at the root of the repository). Let me know how it goes
chiconguyen2601
commented
6 years ago Great. Thank you so much bro. Let me pull the newest version for continue testing today.
aloysius-pgast
commented
6 years ago By default (as in sample code), watchdog is configured to reconnect if no markets data is received for 1800000ms (30 min). You can change the value in constructor options if you want to have a lower timeout
aloysius-pgast
commented
6 years ago I made a small change in the watchdog implementation to ensure that timeout is detected at most timeout value + 10%. So if you set timeout to 30min, it will be detected at most after 33min. Also updated sample code to display the number of timeouts when logging data every 10min https://pastebin.com/JBUB5vqP
chiconguyen2601
commented
6 years ago Okay bro. For now, this version's still receiving updates (after 5h). Thank you very much.
aloysius-pgast
commented
6 years ago Great, keep me updated. I'll also keep monitoring for a few hours and will push a new version to npm after that
aloysius-pgast
commented
6 years ago After 8 hours, still getting data (got 7 reconnections due to timeouts). I wonder if the reason could be the fact that we subscribed to =~ 250 markets. Anyway, pb seems to be partially solved with the watchdog
chiconguyen2601
commented
6 years ago Yes, for now, after 14h, my machine's still getting updates. I think you can push this version to npm. I will use it for my first production, then keep monitoring everyday, if I encounter any problem, I will report to you. Thank you very much.
aloysius-pgast
commented
6 years ago Hi. Everything's fine ? Let me know if I can close issue
chiconguyen2601
commented
6 years ago Yes, it's running ok until now. I think we can close issue right now. Thank you so much.
chiconguyen2601
commented
6 years ago Hi, I have one more question that: what's different between events 'orderBook' and 'orderBookUpdate'?
aloysius-pgast
commented
6 years ago orderbookUpdate only contains the differences from previous event. orderbook is a full orderbook which is sent upon reconnection or when detecting that some update were missed.
chiconguyen2601
commented
6 years ago If so, it means that, whenever I receive a 'orderBook' data, I need to replace existed orderbook on client app, right?
aloysius-pgast
commented
6 years ago Yes
chiconguyen2601
commented
6 years ago Ok, thank you so much bro. I close this issue right now.
Hi Guy,
I use function client.subscribeToMarkets(["BTC-1ST", "BTC-2GIVE"]. But after awhile, I don't receives any data from event ".on('trades'). It seems to be unsubscribed from market (bittrex). How to detect this event?
I checked and saw that the connection is still kept, but can not receive any data from?
Could you help me explain?
Thank you very much.