n0mad01 / node.bittrex.api

No longer maintained
MIT License
183 stars 187 forks source link

Errorr Message: Negotiate Unknown, 503 #67

Closed LRonHubs closed 7 years ago

LRonHubs commented 7 years ago

Error Message: Negotiate Unknown Exception: undefined Error Data: 503

I get this error when trying to connect to WebSockets. Is this something on bittrex's end? If so is it a permanent thing?

ale316 commented 7 years ago

Getting this too. They said they were attacked (https://twitter.com/ramikawach/status/906184766969356289), which prompted them to CloudFlare all their endpoints, including the signalr ones.

zduerr commented 7 years ago

is there a workaround for this?

LRonHubs commented 7 years ago

Spam the PUSH api i guess

masalinas commented 7 years ago

@LRonHubs what do you mean with Spam the PUSH api??

TimLaMason commented 7 years ago

I think he means using the Rest API for grabs. I'd be careful of overuse, as it may get your key banned though.

On Sep 8, 2017 2:17 PM, "algoscale1" notifications@github.com wrote:

facing same issue

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/n0mad01/node.bittrex.api/issues/67#issuecomment-328177334, or mute the thread https://github.com/notifications/unsubscribe-auth/AGJjzBEEg7dtX5HasLnB-CPOr5whi8G5ks5sgYSrgaJpZM4PRah8 .

masalinas commented 7 years ago

@TimLaMason I was using the API more than one week and I never received any error, or banned ... meaby is it a temporal problem?? I created other key and the error is the same

junkomatic commented 7 years ago

Then endpoint is giving '503 - Service Temporarily Unavailable'. I presume they will reactivate it later, or else enable functionality for account auth headers. I have an alert set for if it reactivates normally. Keep us posted if anyone here's anything.

TimLaMason commented 7 years ago

I mean using the rest api for frequent requests in place of the websocket, if it is down. Most api's will ban your key if you make requests more than one every ten seconds or so (just an example). I'm not sure exactly what bittrex's policy is. I'm just saying for those who are thinking of "Spaming the rest api" in place of the signalr hub, it may not be a good idea.

On Sep 8, 2017 2:26 PM, "Miguel Angel Salinas Gancedo" < notifications@github.com> wrote:

@TimLaMason https://github.com/timlamason I was using the API more than one week andI never receive any error, or banned ... meaby is it a temporal problem??

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/n0mad01/node.bittrex.api/issues/67#issuecomment-328179526, or mute the thread https://github.com/notifications/unsubscribe-auth/AGJjzJHAv3r1E8-kpBVCefLZNKfMkrLmks5sgYbNgaJpZM4PRah8 .

masalinas commented 7 years ago

How can I receive the complete error from bittrex?

junkomatic commented 7 years ago

SignalR websocket hubs we're primarily designed for .net. if you code a c# hub client it outputs the error message, and you can also enable tracing to see the raw feed.

masalinas commented 7 years ago

My client I have this code:

// listen to Bittrex WebSocket
    var websocketsclient = bittrex.websockets.listen( function( data ) {
        if (data.M === 'updateSummaryState') {
            data.A.forEach(function(data_for) {
                app.io.emit('bittrex-event', data_for.Deltas);
            });
        }
    });

    websocketsclient.serviceHandlers.connectFailed = function(error) {
        console.log("Websocket connectFailed: ", error);
    };

    websocketsclient.serviceHandlers.onerror = function(error) {
        console.log("Websocket error: ", error);
    };

    websocketsclient.serviceHandlers.connectionLost = function(error) {
        console.log("Connection Lost: ", error);
    };

But not receive any erro detail only the:

Error Message: Negotiate Unknown Exception: undefined Error Data: 503

But where is the callback that receive this error??? how can I get this error in my code???

masalinas commented 7 years ago

@junkomatic how do you code for receive the error: '503 - Service Temporarily Unavailable'? because i only reveive the error: Error Data: 503

junkomatic commented 7 years ago

503 is an http error response code that literally means "Service Temporarily Unavailable". There's not much more data than that.

LRonHubs commented 7 years ago

SignalR websocket hubs we're primarily designed for .net. if you code a c# hub client it outputs the error message, and you can also enable tracing to see the raw feed.

On Sep 8, 2017 3:16 PM, "Miguel Angel Salinas Gancedo" < notifications@github.com> wrote:

@junkomatic https://github.com/junkomatic how do you code for receive the error: '503 - Service Temporarily Unavailable' because i only reveive the error: Error Data: 503

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/n0mad01/node.bittrex.api/issues/67#issuecomment-328190944, or mute the thread https://github.com/notifications/unsubscribe-auth/AWRK7NkCHsuCJKKL87P3i_JrD2mPHhYCks5sgZKdgaJpZM4PRah8 .

ale316 commented 7 years ago

@masalinas You can go into the signalr-client node module and print the request out.

Error Message: Negotiate Unknown
Exception: undefined
Error Data: 503

Is just a catch all error.

What Bittrex is returning is the CloudFlare HTML page saying that they're verifying your connection. I believe the only thing we can do is wait.

n0mad01 commented 7 years ago

i've created a pull request for the signalR lib to at least fix the pass of binding errors into the onerror serviceHandler. https://github.com/mwwhited/signalr-client-nodejs/pull/20

the 503 issue i haven't looked more closely yet.

strelov1 commented 7 years ago

Hmm, why using this service https://www.websocket.org/echo.html WS bittrex is available?

AGoetzDev commented 7 years ago

As already mentioned, the websocket endpoint is currently protected by cloudflare's anti bot system. It is possible to bypass it, by executing a request to the negotiate endpoint (or any page on bittrex really) just as a regular browser would and letting cloudflare do its thing (how cloudflare's bot protection works is explained here: link) then grab the identity cookie and the clearance cookie and send those as the Cookie header together with the User-Agent header (which must be the same as in the request used to obtain the cookies) with the negotiate and websocket connect request. As an example, for python cloudflare-scrape does just that.

ThePrimeagen commented 7 years ago

@AlxGDev how is that a solution? My bot has a lot of responsibility and this really worries me that I could just be taken offline for several hours...

usmankhanic commented 7 years ago

Does anyone know if bittrex have commented on this?

ninesalt commented 7 years ago

Need a fix for this asap

TimLaMason commented 7 years ago

There is also a .net fix:

https://github.com/elcattivo/CloudFlareUtilities

Similar to python's cfscrape. This may help.

On Sep 9, 2017 7:23 AM, "Swailem95" notifications@github.com wrote:

Need a fix for this asap

— You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/n0mad01/node.bittrex.api/issues/67#issuecomment-328271117, or mute the thread https://github.com/notifications/unsubscribe-auth/AGJjzLq13g2upPlxv4gFI7lKHczR13gGks5sgnUkgaJpZM4PRah8 .

metachris commented 7 years ago

@AlxGDev thanks for your suggestion! It also worked for me.

For anyone interested: the goal is to add your cookie and user-agent headers to the signalR websockets connection here: https://github.com/mwwhited/signalr-client-nodejs/blob/master/WhitedUS.SignalR/WhitedUS.signalR/signalR.js#L295

I copied the bittrex and signalr node modules intop my project and modified them a little to achieve this. But you could also edit the signalr-client file directly (not recommended, but the quickest way).

bigdataboss commented 7 years ago

Just replace connectws() function. For the future we could add headers opt. To get cookie without browser you can use nightmare.js

var connectws = function(callback) {
    wsclient = new signalR.client(
      opts.websockets_baseurl,
      opts.websockets_hubs, 
      undefined, true
    );
    wsclient.headers['User-Agent'] = 'YOUR BROWSER UA';
    wsclient.headers['cookie'] = 'cf_clearance=YOUR CF_CLEARENCE FROM BROWSER';
    wsclient.start(); 
    wsclient.serviceHandlers = {
      bound: function() {
        ((opts.verbose) ? console.log('Websocket bound') : '');
      },
      connectFailed: function(error) {
        ((opts.verbose) ? console.log('Websocket connectFailed: ', error) : '');
      },
      disconnected: function() {
        ((opts.verbose) ? console.log('Websocket disconnected') : '');
      },
      onerror: function(error) {
        ((opts.verbose) ? console.log('Websocket onerror: ', error) : '');
      },
      bindingError: function(error) {
        ((opts.verbose) ? console.log('Websocket bindingError: ', error) : '');
      },
      connectionLost: function(error) {
        ((opts.verbose) ? console.log('Connection Lost: ', error) : '');
      },
      reconnecting: function(retry) {
        ((opts.verbose) ? console.log('Websocket Retrying: ', retry) : '');
        // change to true to stop retrying
        return false;
      }
    };
    return wsclient;
  };
0b01 commented 7 years ago

@cyberwlf I did exactly that. But it's still not working for me...

bigdataboss commented 7 years ago

@rickyhan double check everything. If Bittrex works in your browser and you copied user-agent and the cookie correctly it should work.

ale316 commented 7 years ago

@rickyhan Hey, I just got it to work on my server. Make sure that the IP you used to get the cf_clearance is the same as the one making the subsequent requests.

For the server I used this python library (https://libraries.io/github/Limero/cloudflare-scrape), to grab User-Agent and cf_clearance cookie, then did what @cyberwlf suggested (thanks!) and it worked.

0b01 commented 7 years ago

@cyberwlf @ale316 Thanks, I'll try that on my IPv4 server. My home address uses IPv6.

Update: It's working! Thank you! It's indeed because of IPv6.

0xecute commented 7 years ago

Did you make it work with python? I got the correct cookie and the correct user agent header, but I got the following error from signalr-client: error: There was an error invoking Hub method 'corehub.SubscribeToExchangeDeltas'.

Thanks for your help

ale316 commented 7 years ago

@0xecute If it throws that error, it is connected, so it seems to not be an issue with Cloudflare's protection. I guess make sure you're invoking the call correctly.

0xecute commented 7 years ago

Thanks, I found my mistake, nothing to do with cloudflare :)

KevinMcGovern commented 7 years ago

@rickyhan Had the same issue, it was IPv6. For anyone else, if you're running locally on a Mac you can enter into Terminal:

networksetup -setv6off Wi-Fi to Disable IPv6

and

networksetup -setv6automatic Wi-Fi to re-enable it later.

I disabled IPv6, refreshed Bittrex to get a new cf_clearance cookie and it works perfectly now.

armandohg commented 7 years ago

You can use this NodeJS package to get the cf_clearance code: https://github.com/codemanki/cloudscraper

create new js file: vi scraper.js var cloudscraper = require('cloudscraper'); cloudscraper.get('https://bittrex.com/', function(error, response, body) { if (error) { console.log('Error occurred'); } else { console.log(body, response); } });

run it: node scraper.js

and it will throw your User Agent and cf_clearance code

then use cyberwlf's method

usmankhanic commented 7 years ago

@armandohg - Can you post the exact source code? many thanks - would be much appreciated

armandohg commented 7 years ago

If you are using the Bittrex NodeJS package then you know how to install other packages, right?

Just create a new JS script with the code I pasted (var cloudscraper = require('clo....). This script will throw a lot of info, including a User Agent and a cf_clearance code.

Use these two pieces and paste them in cyberwlf's code.

If you still need help, let me know.

reeZee3U commented 7 years ago

In Chrome browser open Inspector -> Network, then refresh the page https://bittrex.com/account/login and check request headers to login, now you can see your cf_clearance and user-agent.

thanhlm86 commented 7 years ago

@armandohg help me, i run node scraper.js

{ errorType: 0, error: { Error: tunneling socket could not be established, cause=getaddrinfo ENOTFOUND 3128 3128:8 0 at ClientRequest.onError (C:\Users\ThinkPad\Desktop\node-bittrex-api\node_modules\tunne l-agent\index.js:177:17) at Object.onceWrapper (events.js:316:30) at emitOne (events.js:115:13) at ClientRequest.emit (events.js:210:7) at Socket.socketErrorListener (_http_client.js:401:9) at emitOne (events.js:115:13) at Socket.emit (events.js:210:7) at emitErrorNT (internal/streams/destroy.js:64:8) at _combinedTickCallback (internal/process/next_tick.js:138:11) at process._tickCallback (internal/process/next_tick.js:180:9) code: 'ECONNRESET' } }

mjmau commented 7 years ago

Thanks to @cyberwlf and @armandohg, combining your two fixes works great!

bigdataboss commented 7 years ago

Now let's combine them! Your connectws():

var connectws = function(callback) {
    wsclient = new signalR.client(
      opts.websockets_baseurl,
      opts.websockets_hubs, 
      undefined, true
    );
    if (opts.headers) {
      wsclient.headers['User-Agent'] = opts.headers.user_agent;
      wsclient.headers['cookie'] = opts.headers.cookie;
    }
    wsclient.start(); 
    wsclient.serviceHandlers = {
      bound: function() {
        ((opts.verbose) ? console.log('Websocket bound') : '');
      },
      connectFailed: function(error) {
        ((opts.verbose) ? console.log('Websocket connectFailed: ', error) : '');
      },
      disconnected: function() {
        ((opts.verbose) ? console.log('Websocket disconnected') : '');
      },
      onerror: function(error) {
        ((opts.verbose) ? console.log('Websocket onerror: ', error) : '');
      },
      bindingError: function(error) {
        ((opts.verbose) ? console.log('Websocket bindingError: ', error) : '');
      },
      connectionLost: function(error) {
        ((opts.verbose) ? console.log('Connection Lost: ', error) : '');
      },
      reconnecting: function(retry) {
        ((opts.verbose) ? console.log('Websocket Retrying: ', retry) : '');
        // change to true to stop retrying
        return false;
      }
    };
    return wsclient;
  };

Your main code:

const bittrex = require('node.bittrex.api');
const cloudscraper = require('cloudscraper');

cloudscraper.get('https://bittrex.com/', function (error, response, body) {
    if (error) {
        console.log('Cloudscraper error occurred');
    } else {  
        bittrex.options({
            headers: {
                cookie: response.request.headers["cookie"],
                user_agent: response.request.headers["User-Agent"]
            }
        });
        console.log('CloudFlare nailed');
        start(); // Your code here
    }
});
ninesalt commented 7 years ago

Can we expect an official update to the module soon though?

bigdataboss commented 7 years ago

@Swailem95 Take a look: https://github.com/n0mad01/node.bittrex.api

ninesalt commented 7 years ago

Well that sucks. Maybe someone's going to open a PR and fix it himself. That would be great.

glaucoheitor commented 7 years ago

Just need to update you code @cyberwlf, because response.request.headers["cookie"] replies with __cfduid AND cf_clearance.

bigdataboss commented 7 years ago

@glaucoheitor It's a hot fix, not an official solution. BTW, what's the problem with __cfduid ? It seems working

glaucoheitor commented 7 years ago

I know it's a hotfix. I am trying to use this here too. Here don't work with that code. but if I put the cf_clearence code directly to signalR it works

bigdataboss commented 7 years ago

@glaucoheitor Works on my side. I don't see any reason why __cfduid could interrupt the process. You can update my solution btw :)

glaucoheitor commented 7 years ago

That what's weird, cause it works directly on the module file.

I was trying to update your solution, but I dont have that much knowledge. lol

bigdataboss commented 7 years ago

@glaucoheitor double check that you copied the exact code. Especially connectws() function. I use new "headers" option

thanhlm86 commented 7 years ago

Any body help,

i try the way @armandohg, it show error :

{ errorType: 0, error: { Error: tunneling socket could not be established, cause=getaddrinfo ENOTFOUND 3128 3128:80 at ClientRequest.onError (C:\Users\ThinkPad\Desktop\node-bittrex-api\node_modules\tunnel-agent\index.js:177:17) at Object.onceWrapper (events.js:316:30) at emitOne (events.js:115:13) at ClientRequest.emit (events.js:210:7) at Socket.socketErrorListener (_http_client.js:401:9) at emitOne (events.js:115:13) at Socket.emit (events.js:210:7) at emitErrorNT (internal/streams/destroy.js:64:8) at _combinedTickCallback (internal/process/next_tick.js:138:11) at process._tickCallback (internal/process/next_tick.js:180:9) code: 'ECONNRESET' } } how to fix problem? Thanks.

dparlevliet commented 7 years ago

For those looking for an integrated fix for this issue I have committed it to https://github.com/dparlevliet/node.bittrex.api

Important note: this makes the websocket connection asynchronous so please see the updated documentation https://github.com/dparlevliet/node.bittrex.api#websockets

There is a unit test added which you can test with npm test tests/websocket.js to ensure it works for you.

If you have any further comments or suggestions please add them to here https://github.com/dparlevliet/node.bittrex.api/issues/1