Cloudflare CAPTCHA on API requests

[dbose]

Being greeted with Cloudflare API protection screen while making private API requests.

POST https://api.btcmarkets.net/order/history

Headers

Accept-Charset:UTF-8 apikey:xxx Accept:application/json signature:xxx timestamp:xxx Content-Type:application/json

Body

'{"currency":"AUD","instrument":"ETH","limit":10,"since":null}'

Using following to sign requests

sign: function(path, api, method, params, headers, body) {
    var uri = '/' + utils.implodeParams (path, params);
    var url = urls['api'][api] + uri;
    if (api === 'private') {
      var nonce = utils.nonce().toString ();
      var auth = undefined;
      headers = {
        'apikey': apiCredentials.apiKey,
        'timestamp': nonce,
      };
      if (method === 'POST') {
        headers['Accept'] = 'application/json';
        headers['Content-Type'] = 'application/json';
        headers['Accept-Charset'] = 'UTF-8';
        auth = uri + "\n" + nonce + "\n"; // eslint-disable-line quotes
        body = JSON.stringify(params);
        auth += body;
      } else {
        var query = utils.keysort (utils.omit (params, utils.extractParams (path)));
        var queryString = '';
        if (Object.keys (query).length) {
          queryString = encodeURIComponent(query);
          url += '?' + queryString;
          queryString += "\n"; // eslint-disable-line quotes
        }
        auth = uri + "\n" + queryString + nonce + "\n"; // eslint-disable-line quotes
      }
      var signature = utils.hmac(auth, apiCredentials.secret);
      headers['signature'] = signature;
    } else {
      if (Object.keys (params).length) {
        url += '?' + encodeURIComponent (params);
      }
    }
    return { 'url': url, 'method': method, 'body': body, 'headers': headers };
  }

hmac: function(request, secret){
    var result = Utilities.computeHmacSignature(Utilities.MacAlgorithm.HMAC_SHA_512,request,secret);
    return Utilities.base64Encode(result);
  }

[dbose]

Does it help to whitelist my IP through https://www.projecthoneypot.org/white_list.php ?

[dbose]

I understand the WAF stuff, but there has to be way to allow legitimate API users without getting shutdown by IP reputation stuff. What happens if my IP is dynamic ?

[martin-nginio]

Hi @dbose

Thank you for your feedback and being patience.

We do not have any blacklisting in general for any particular client and also your API request seems to be reasonbale.

It's possible that the request is being blocked by CloudFlare (our front face network ) and for our team to be able to do any further troubleshooting, we'd need to get your current ip and the other info like the country you are making this request from.

As this is a public Github for coding discussions, please can I ask you to submit a request with our support team and please share your current ip, isp, and also country, etc.
https://support.btcmarkets.net/hc/en-us

Thanks.

[dbose]

Thanks @martin-nginio

Have already filed a ticket to support and I've been asked to report the issue here. Will file a ticket again.

[martin-nginio]

My apologies if there has been miscommunication on this issue.

We will follow this up in our end and will get back to you. Opening a support ticket is helpful as it allows us to capture private information as well (e.g. your ip address, location, etc)

Thanks.

Regards, Martin

[dbose]

Thanks Martin.

Let me know any updates/ETA on this issue.

Thanks Deb

On Fri, 30 Aug 2019 at 4:10 pm, martin-nginio notifications@github.com wrote:

My apologies if there has been miscommunication on this issue.

We will follow this up in our end and will get back to you. Opening a support ticket is helpful as it allows us to capture private information as well (e.g. your ip address, location, etc)

Thanks.

Regards, Martin

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/BTCMarkets/API/issues/169?email_source=notifications&email_token=AADHS37IBKN2AP4QWEYV2KDQHC2VPA5CNFSM4IR6ALHKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5QVCIA#issuecomment-526471456, or mute the thread https://github.com/notifications/unsubscribe-auth/AADHS35RYMLVSXW23ZBVJX3QHC2VPANCNFSM4IR6ALHA .

[martin-nginio]

Hi @dbose

Given information you shared via private support ticket, I can confirm CloudFlare blocks your request as it's detecting possible SQL injection in the http request which needs further investigation.

It's possible that your api client code or the infrastructure running it (and I have not tested it) generates a http request with some special characters inside body or header that is considered abnormal by CloudFlare.

I'd suggest looking at our very basic sample node.js client app here and compare request details: https://github.com/BTCMarkets/api-client-node

Thanks.

Regards, Martin

[martin-nginio]

One more notes. In case if you are just starting to develop your trading app, then I'd suggest checking out our new generation API that we are about to release: https://api.btcmarkets.net/doc/v3

This API is in beta during September so it's not released yet but it has more features and capabilities and we expect it'd be easier to integrate with it. If you have questions/feedbacks please let us know.

By the way, consuming the new API v3 (or exiting one) should not impact CloudFlare issue which seems to be separate.

Thanks for your support.

Regards, Martin

[dbose]

Hi Martin,

This is fantastic. Thanks for the sample client to compare generated requests. BTCMarkets rock!

Thanks Deb

On Wed, Sep 4, 2019 at 12:28 PM martin-nginio notifications@github.com wrote:

One more notes. In case if you are just starting to develop your trading app, then I'd suggest checking out our new generation API that we are about to release: https://api.btcmarkets.net/doc/v3

This API is in beta during September so it's not released yet but it has more features and capabilities and we expect it'd be easier to integrate with it. If you have questions/feedbacks please let us know.

By the way, consuming the new API v3 (or exiting one) should not impact CloudFlare issue which seems to be separate.

Thanks for your support.

Regards, Martin

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/BTCMarkets/API/issues/169?email_source=notifications&email_token=AADHS34CUTT7ATBS7KL4OCLQH4MLXA5CNFSM4IR6ALHKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD52EE4I#issuecomment-527712881, or mute the thread https://github.com/notifications/unsubscribe-auth/AADHS37BKIURXN7WW3FRLYTQH4MLXANCNFSM4IR6ALHA .

[dbose]

Unrelated question:

Have you considered building an algorithmic front end with Quantopian components (https://www.quantopian.com/opensource) over BTCMarkets. I'm planning to build something exclusive for BTCMarkets.

Thanks Deb

On Wed, Sep 4, 2019 at 5:47 PM Deb Bose bose.debasish@gmail.com wrote:

Hi Martin,

This is fantastic. Thanks for the sample client to compare generated requests. BTCMarkets rock!

Thanks Deb

On Wed, Sep 4, 2019 at 12:28 PM martin-nginio notifications@github.com wrote:

One more notes. In case if you are just starting to develop your trading app, then I'd suggest checking out our new generation API that we are about to release: https://api.btcmarkets.net/doc/v3

This API is in beta during September so it's not released yet but it has more features and capabilities and we expect it'd be easier to integrate with it. If you have questions/feedbacks please let us know.

By the way, consuming the new API v3 (or exiting one) should not impact CloudFlare issue which seems to be separate.

Thanks for your support.

Regards, Martin

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/BTCMarkets/API/issues/169?email_source=notifications&email_token=AADHS34CUTT7ATBS7KL4OCLQH4MLXA5CNFSM4IR6ALHKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD52EE4I#issuecomment-527712881, or mute the thread https://github.com/notifications/unsubscribe-auth/AADHS37BKIURXN7WW3FRLYTQH4MLXANCNFSM4IR6ALHA .

[martin-nginio]

Hi @dbose ,

We, as a company do not run any trading bots or trading algorithms on our trading platform but if you are working on using Quantopian to run trading strategies it's great idea and we'd be interested to see how you go. We can help listing it on our API page for other customers (if you decide to open source it)

Thanks for your support.

Regards, Martin

[martin-nginio]

Hi @dbose

I'm closing this issue for CloudFlare interceptions in your API requests. Please let us know if you still experience issues with CloudFlare.

Also, we are interested to hear from you regarding your trading application using Quantopian so please feel free to open up new ticket or contact us directly via our support portal and we will pick up the conversation from there.

Thank you again for your support and feedback.

Regards, Martin