BTCMarkets / API

API
119 stars 30 forks source link

Transaction history endpoint supports max 100 transactions per query. #168

Closed MaxPovver closed 5 years ago

MaxPovver commented 5 years ago

Hello from Cointracking.info

We are adding you to our website, and Transaction api seems to be the best for our needs. https://github.com/BTCMarkets/API/wiki/Transaction-API However, it ignores all page sizes higher than 100, while in api docs it says default page size will be 200. Is it possible to somehow set at least 1000 page size? That would increase trading history import speed for our users a lot.

martin-nginio commented 5 years ago

Hi @MaxPovver

Thanks for your feedback.

I can confirm this is a mistake in our side and the max limit should be 200. We will fix this in the coming days.

We have used 200 as a generic guide across our system and whilst we can increase this number however we always suggest smaller payloads for API calls and more frequent API calls where possible instead of big payloads.

Please note, this week we have just increased rate limiting of all of our APIs and the default (including this particular API) is 50 calls per 10 seconds which I understand is reasonable enough for all of the APIs.

Also, transactions are only generated when there is fund transfer or trade execution activities and this particular API is used for historical reasons and in most cases a single call per minute might suffice even for 100 max results ( you still have the ability to call in much higher rate).

If you are after low latency data for trading, then I'd suggest calling order/trade related APIs as most of them (e.g. open orders, order detail, cancelall, etc) would give you data with lower latency.

If you have specific use cases, please feel free to cover them here (this is a public site) or raise a ticket with our support team so we can better help.

I hope this helps.

Thanks again.

Regards, Martin

MaxPovver commented 5 years ago

Hi @martin-nginio

Thank you for such a quick response!

We don't care about latency because of our use case.

What Cointracking does is importing all user's trades for all currency pairs & all deposits and withdrawals, helping them to analyze balance changes.

So we do not do anything trading related, just load full account balance changes data. It would be great to import all user's trades for all currency pairs at once, without specifying specific trading pairs(like in Transaction API), that would make us to iterate over all of them, is that possible? I couldn't find such method in api docs.

Best regards, Maksim Chistov

MaxPovver commented 5 years ago

Looks like using Transactions API for downloading trade history will be painful because reference id is the same for different trades: [{"id":3220644305,"creationTime":1556020262865,"description":"Buy 0.57693557ETH @ AUD 247.14000000 Trading fee","currency":"AUD","balance":...,"amount":-121196135,"recordType":"Trade","referenceId":3220643999,"action":"Trading Fee"},{"id":3220644304,"creationTime":1556020262864,"description":"Buy 0.57693557ETH @ AUD 247.14000000 Trade settled","currency":"ETH","balance":....,"amount":57693557,"recordType":"Trade","referenceId":3220643999,"action":"Buy Order"},{"id":3220644303,"creationTime":1556020262864,"description":"Buy 3.87693560ETH @ AUD 247.14000000 Fully matched at 247.14000000","currency":"AUD","balance":....,"amount":-14258385676,"recordType":"Trade","referenceId":3220643999,"action":"Buy Order"},{"id":3220644088,"creationTime":1556020260077,"description":"Buy 1.30000000ETH @ AUD 247.14000000 Trading fee","currency":"AUD","balance":....,"amount":-273089378,"recordType":"Trade","referenceId":3220643999,"action":"Trading Fee"},{"id":3220644087,"creationTime":1556020260076,"description":"Buy 1.30000000ETH @ AUD 247.14000000 Trade settled","currency":"ETH","balance":....,"amount":130000000,"recordType":"Trade","referenceId":3220643999,"action":"Buy Order"},{"id":3220644086,"creationTime":1556020260076,"description":"Buy 3.87693560ETH @ AUD 247.14000000 Partially matched at 247.14000000","currency":"AUD","balance":....,"amount":-32128200000,"recordType":"Trade","referenceId":3220643999,"action":"Buy Order"},{"id":3220644051,"creationTime":1556020259812,"description":"Buy 2.00000000ETH @ AUD 247.11000000 Trading fee","currency":"AUD","balance":.....,"amount":-420086505,"recordType":"Trade","referenceId":3220643999,"action":"Trading Fee"},{"id":3220644050,"creationTime":1556020259811,"description":"Buy 2.00000000ETH @ AUD 247.11000000 Trade settled","currency":"ETH","balance":.....,"amount":200000000,"recordType":"Trade","referenceId":3220643999,"action":"Buy Order"},{"id":3220644049,"creationTime":1556020259811,"description":"Buy 3.87693560ETH @ AUD 247.14000000 Partially matched at 247.11000000","currency":"AUD","balance":....,"amount":-49422000000,"recordType":"Trade","referenceId":3220643999,"action":"Buy Order"}]

martin-nginio commented 5 years ago

Thanks for describing your use case. It's a better option to provide you with ability to download the full transaction report for a user rather than you having to call for multiple wallets, etc.

Let me discuss with our team and will get back to you.

Please note, our website already have such facility so it' d be great if you can take a look at the transaction history export on the website (it's under the Account menu, transaction history) to make sure the data works for you.

note:

Please can you take a look and let us know if the data makes sense to you. We can probably also provide the data in json format if it's preferred but I just want to make sure if it works.

Thanks.

MaxPovver commented 5 years ago

Hi, Martin

The way you offer is totally fine for us, as for csv format, I don't have account on btcmarkets, but if it has deposits, withdrawals, trades & fees and every transaction there has some unique id, that is okay. Having json format is not necessary, csv is also fine, although json is preferred because it allows to send both data & status like sucess/error

Best regards, Maxim Chistov

MaxPovver commented 5 years ago

UPD: I've checked csv headers, and it seems to work the same way as transactions endpoint, is that right? In that case it is not very good because it has more than one transaction per one trade which makes importing them complicated. We prefer format described here: https://cointracking.freshdesk.com/a/solutions/articles/29000022558-integrating-a-new-exchange-api-with-cointracking-info

MaxPovver commented 5 years ago

Hey, @martin-nginio , was there any progress on this? Thanks!

martin-nginio commented 5 years ago

Hi @MaxPovver

Thanks for your patience.

The transaction report covers every single changes to all wallets involved in an operation (e.g. trade execution, fund transfer). For instance, if you place an order to buy BTC paying via AUD (assuming a single trade executes), there will be the following 3 records as separate items:

  1. AUD amount paid (as a change to AUD wallet)
  2. BTC amount received
  3. Fee paid (in this case in AUD currency)

All of those records have individual record ids.

The current API provides all of those 3 records for a given order. The csv report covers the same data plus better categorization of those individual items and also a single orderId that you can use to group them together in a single raw.

Please let me know if this covers what you need. I'd suggest signing up with BTC Markets and looking at the data as it helps better understanding the data format.

Thanks.

Regards, Martin

MaxPovver commented 5 years ago

@martin-nginio that looks good enough for our case. Althoug there is one problem - sometime order executes with more than one trade, and in this case there 3 * N transactions (if there were two trades, there will be 9 transaction, 3 for each trade and 3 for total). But I guess you can't change that anyway. So waiting for your new api endpoint.

martin-nginio commented 5 years ago

ok. thanks for clarifying. We will make the report API available this week.

martin-nginio commented 5 years ago

Hi @MaxPovver

Please find below a simple guide to use the report API which allows you to generate and access same transaction report that is available via the website. This new API is part of our v3 API generation and is still in beta but it should be relatively stable. Please take a look and if you come across any issues please let us know.

https://github.com/BTCMarkets/api-v3-doc/blob/master/reports.md

Thanks.

MaxPovver commented 5 years ago

@martin-nginio thank you, we will add it soon.

MaxPovver commented 5 years ago

@martin-nginio do you support readonly API keys? That allow only getting trades info, but NOT trading and NOT withdrawing. I can't check that because you have to verify account before creating api keys

MaxPovver commented 5 years ago

Ok I found https://github.com/BTCMarkets/API/issues/103 :)

martin-nginio commented 5 years ago

Hi @MaxPovver yes they are available on the website and I'm glad you found it already.

By the way, over the next few days we will update API v3 beta and this update will break the Authentication due to change in the name of the http headers. I will send you the new name of the http headers so you can also update your client code.

I'm sorry for the inconvenience as this requires you to change in your code so it will continue to work.

Thanks for your feedback and support again. We are making progress towards a public availability of the API v3 soon.

Regards, Martin

martin-nginio commented 5 years ago

Hi @MaxPovver

Just letting you know that API has been updated today which is a breaking change for your current code (as at the time we were using the old authentication mechanism in our end).

Most of the process is the same except for the followings:

  1. Use the new header names: BM-AUTH-APIKEY, BM-AUTH-TIMESTAMP and BM-AUTH-SIGNATURE
  2. Use the following combination to build string for signature: method + path + timestamp + data

You can read further here: https://api.btcmarkets.net/doc/v3#section/Authentication/Authentication-parameters

Sorry again for the breaking change. This week we are going beta with the new API hence it's less likely for such changes.

Thanks again for your support.

Regards, Martin

MaxPovver commented 5 years ago

Hey @martin-nginio . We've updated for your breaking change

MaxPovver commented 5 years ago

So we've done some imports so far. @martin-nginio do you see any issues from your side?

martin-nginio commented 5 years ago

Thanks @MaxPovver

We have not seen any issues in our end but there was a one of our customers using your service but who contacted our support regarding the the following error yesterday using CoinTracking (the full error message includes api key that I have removed here): (Error: Report id was not returned.)

As we don't seem to generate this particular error message, would you be able to also take a look and let me know if you come across issues as well?

Potentially, this could be a case where the report was not prepared within the time period (e.g. 30 seconds) and you may need to re-try again to get the report by id (or message the customer to re-try depending on your integration logic).

If you are aware of any particular API failure please also let me know and we will investigate further.

Thanks for the follow up.

Regards, Martin

MaxPovver commented 5 years ago

@martin-nginio that was wrong handling of incorrect api keys error on our side. It is fixed now. It would be great if you'd have some "status" field, so we would know that if it is "false" there would be "error_message" field that would show human readable error that we could show to our users on any error and stop trying to get the report.

MaxPovver commented 5 years ago

like {"status":"ok", "reportId":"3232323232"} or {"status":"error", "error_message":"wrong signature"} or {"status":"error", "error_message":"server is overloaded. please try again later"}

martin-nginio commented 5 years ago

I assume you are referring to the API that generates a report. /v3/report with POST

We have tried to stick to REST API general guidelines so the error handling at the high level is done based on http response status that comes with a response body that represents more detail about the error.

For instance, if there is an issue with authentication (e.g. signature or api key, etc) then you should get http 401, 403 and along with that the response body has the following body already: {"code":"InvalidAPIKey","message":"invalid api key"} and this can be used to message users about what went wrong.

This is the same pattern for situations where the request is consider to be http 400 bad request so in those cases also response body would have this:
{"code":"InvalidReportParameter","message":"invalid report format"}, etc

A successful http call to any of the APIs returns http 200 along with an entity like below as the response body: {"reportId":"123456789"}

Hope this clarifies the error handling.

There is also covered https://api.btcmarkets.net/doc/v3#section/Error-handling

If any of the specific APIs does not return proper detail error response pls let us know and we will fix them quickly as we are getting close to release.

Thanks.

Regards, Martin

martin-nginio commented 5 years ago

Hi @MaxPovver

I'm following up with regards to the integration and see if you are experiencing any issues. One of our customers reported that they see partial records in CoinTracker (no records since 5th of September) and I'm wondering if everything is running ok.

Thanks.

Regards, Martin

martin-nginio commented 5 years ago

Also as our customers user your service, I tried to register on Cointracking.info site but was not able to add my BTC Market's API key. Please can you confirm if the integration is working and you still support BTC Markets. Thanks.

MaxPovver commented 5 years ago

Hey, @martin-nginio Integration seems to work fine. Some users complained about problems with import but none yet have sent us their api keys. To use api imports you should have paid account, but for you we can create test account with aoi import support. Please write to dario@cointracking.info about that and cc my mc@maxpower.ee

martin-nginio commented 5 years ago

thanks @MaxPovver for your reply.

I just created a paid account and will connect with you directly regarding user issues.

Will also close this ticket for now. If anything needs troubleshooting will create new issues.

Thanks.

Regards Martin