Error caused by Cloudflare landing page

[mapf0000]

Hi,

first of all, thank you for this nice websocket example for bittrex.

It seems that Bittrex is using a cloudflare landing page to protect against DDOS attacks. This causes the following exception in your code:

[Thread-0] WARN bittrex.BittrexWS - webSockets - Error: microsoft.aspnet.signalr.client.http.InvalidHttpStatusCodeException: Invalid status code: 503

full gist: https://gist.github.com/mapf0000/9fdc4541937ec143594c7709bc118f70

Bittrex must have made these changes today, yesterday the code worked flawlessly.

I am trying to work out a fix by myself but i am a beginner when it comes to networking/http, so any help/tip would be welcome.

[andre77]

Hi Max, glad to hear, that my code is used out there. Well, to be honest, i did not notice any bigger problems with the bittrex websocket API. On the other side the server reply status code 503 is an internal server error, which could have been occured. May be there are some maintenance work on the bittrex server side. Who knows, why a server replies with an internal error code :-) Does the error still exist? Anyway your code needs to be able to handle such errors, websocket connections as any other connections may be broken at any time, so you need to catch the errors and implement a reconnect on your side.

[mapf0000]

The error still exists. Using Bittrex from the browser works fine (inspected web socket traffic with firefox). I'm also little unaware why this error message (503) comes up :D

In the server response that causes the error there are the lines: <h1><span data-translate="checking_browser">Checking your browser before accessing</span> bittrex.com.</h1> <p data-translate="process_is_automatic">This process is automatic. Your browser will redirect to your requested content shortly.</p> <p data-translate="allow_5_secs">Please allow up to 5 seconds&hellip;</p>

so my first thought was that the problem is related to handling the redirect.

Will report back when i make some progress.

[andre77]

have a look into the current discussion at the slack channel https://bittrex.slack.com/messages/C3ZGRGLAV looks like, there are actually issues with the bittrex servers

[mapf0000]

Unfortunately i have no access to their slack channel. But if you think the error is caused by bittrex itself you can close this issue.

I will just try it again later this day/tomorrow.

Anyway, thanks for your help.

[mapf0000]

I tested a bit more and don't think its a server error. Bittrex works fine from browser and when using rest requests.

As far as i understand the code, the init() method in the BittrexWS class runs SignalRFuture<Void> start = connection.start(new WebsocketTransport(logger)); start.get(); which somewhere down the line causes a http request to bittrex.com.

Because of the Cloudflare DDOS protection this request is redirected and the 503 error is raised. The response header of this redirect looks like this: Refresh [8;URL=/cdn-cgi/l/chk_jschl?pass=1510496233.229-LI3g2iPHi5] X-Frame-Options [SAMEORIGIN] Transfer-Encoding [chunked] null [HTTP/1.1 503 Service Temporarily Unavailable] CF-RAY [3bca1cf8a9732372-FRA] Server [cloudflare-nginx] Cache-Control [no-cache] Connection [close] Set-Cookie [__cfduid=dd1ccf2de7f0e596eacc85e2bc8cf0dd31510496229; expires=Mon, 12-Nov-18 14:17:09 GMT; path=/; domain=.bittrex.com; HttpOnly] Date [Sun, 12 Nov 2017 14:17:09 GMT] Content-Type [text/html; charset=UTF-8]

I think its required to set the cookie and send a new request with the refresh parameters diplayed in the header.

Now my problem is, i dont know where to start to implement this :D Any suggestions?

[va-prans]

@mapf0000 since you closed this.. did you find a fix? If so would be so kind to share it :D?

[mapf0000]

I closed this because the code as it is would work when bittrex goes out DDOS attack mode.

To make it work you have to solve the Cloudflare DDOS protection page (this means solving a javascript challange and using a valid browserr user agent in the header of the http requests). After this challange is solved you have two cookies (+ the user agent) which have to be set for all http(s) and websocket connections in the signalr library.

You can find my version of a cloudflare solver here (the whole repository is work in progress, so the code isn't overly beautiful): https://github.com/mapf0000/gekko/blob/master/src/main/java/de/gekko/websocketNew/CloudflareScraper.java which is basically a java port of this: https://github.com/Anorov/cloudflare-scrape

I didnt attempt to integrate a fix in this library because I wanted to reverse engineer the protocol used by bittrex myself, so you have to find out how to set cookies in the signalr library yourself.

I hope this is useful for you.

[usmankhanic]

This is still broken - need the cloudflare integrated fix in place, if you want to see traction of downloads - the node api is flakey at best - you have the opportunity to cream it if you can get that cloudflare bit in!

Thanks

[andre77]

@usmankhanic currently i face the same problem, and had no time yet to have a deeper look into it @mapf0000 the link to CloudflareScraper.java does not work, could you please provide another one?

guys, feel free to create a PR for this problem, it would be very appreciated

[mapf0000]

https://github.com/mapf0000/gekko/blob/master/src/main/java/de/gekko/websocket/CloudflareScraper.java

I will post a detailed explaination this evening, on mobile atm. I can build a pullrequest for the scraper with a more useful api for this library (convenience method for getting the relevant cookies).

The main open problem to look into is setting these cookies in the http and websocket client used to connect bittrex.

[andre77]

so, guys, i spent a couple of hours playing around this issue the CloudflareScraper.java provided by @mapf0000 worked partially. there is another project facing the same topic: https://github.com/alessiop86/anti-antibot-cloudflare

any way, its is not really related to this little project, if you ask me. I will not provide the integration of a cloudflare workaround or similar. if anybody would provide a PR, please do so, i will accept it.

The much cleaner way to use the bittrex API without having to solve the cloudflare challenge is to set your IP to a white list, in that case you are exempt from the cloudflare challenge.