search

DigitalRuby / ExchangeSharp

ExchangeSharp is a powerful, fast and easy to use .NET/C# API for interfacing with many crypto currency exchanges. REST and web sockets are supported.
https://www.digitalruby.com
MIT License
737 stars 374 forks source link

Coinbase API changes: #810

Closed Varmetric closed 7 months ago

Varmetric commented 11 months ago

As of August 1, the Coinbase API has changed and requires an authenticated connection. Obviously this API no longer works with Coinbase

https://docs.cloud.coinbase.com/exchange/docs/websocket-auth

BobDeCuir commented 8 months ago

While I don't agree with Coinbase's decision to require authentication on Level2 requests, I'm surprised this hasn't come up before. This is indeed problematic for ExchangeSharp web requests, because all of the socket APIs are behind sealed and private classes/calls which cannot be overridden. Even the ClientWebSocketImplementation (defaults to System.Net) is private, so there's not much point allowing a different implementation that can't be changed.

For Coinbase Orderbook sockets, both the connect and channel request must be with authentication headers, which can't be set from the Coinbase Exchange API directly (or indirectly for that matter.) This will require rethinking the ClientWebSocket API implementation.

vslee commented 8 months ago

FYI, we dont have coinbase support. We have only have it for Coinbase Pro, which they are deprecating in favor of the new Coinbase Advanced Trade. We would be happy to take Pull Requests which add support for Coinbase Advanced Trade, including any modifications of ClientWebSocket.

BobDeCuir commented 8 months ago

Working on V2 Coinbase support at this very moment. Thank you.

O-Mutt commented 7 months ago

@BobDeCuir i don't see any active branches from you and am interested in the new coinbase impl and would love to help. Have you made any substantial progress?

BobDeCuir commented 7 months ago

What a wild ride it has been...

To date, there are FIVE different APIs being offered by Coinbase: 1) The original V1 - "https://exchange.coinbase.com" 2) CoinBase Pro - "https://api.pro.coinbase.com" 3) Advanced Trade V2 - "https://api.coinbase.com/v2" 4) The next gen which they are in the process of converting to but have yet to document 5) The FIX protocol used extensively by banking, but no one else uses

There are also THREE Socket Feeds 1) The Original V1 - "wss://ws-feed.exchange.coinbase.com" 2) Coinbase Pro - "wss://ws-feed.pro.coinbase.com" 3) Advanced Trade V2 - "wss://advanced-trade-ws.coinbase.com"

But wait, there's more...

Many of these separate APIs have different Versions (especially Advanced Trade), which change the JSON return data for many calls. Also, if you've transitioned from the original Coinbase site to Pro, then to Advanced Trade, your trade history remains, but is returned by the API in different formats depending on the origin. This makes parsing a nightmare.

And the hits keep on coming...

If you created an API Key from the original V1 Coinbase, it still works... sort of. The original V1 API is being depreciated, as are most of the Pro APIs. On their site you can specify which API/Version the key is being used for. You can "Upgrade" this to a later version (or latest Advanced Trade) but this is a one-time process and you cannot revert backwards. Most (not all) of the "Public" APIs of Coinbase Pro still work - for now. There are fewer Public APIs (not requiring an API Key) with Advanced Trade.

Advanced Trade Errata...

Pagination is returned for most Get calls as an element in a JArray. In Pro, the pagination was returned in the Response Header. Not a big deal, really, just different.

All calls must be signed, with few exceptions.

Advanced Trade is unique due to the obfuscating reliance on "Account Ids" (which is ridiculous IMO). The Client/Customer/User belonging to any given API Key does not have a single Account Id. They have a separate AccountId for every coin/currency that Coinbase supports. Additional AccountIds can created/deleted for each supported coin.

An AccountId is required when retrieving any history. You want to see all trades for "BTC", for example? Good luck. First you need to know the AccountId. Since there are dozens and possibly hundreds of AccountIds, retrieving these is a separate API call - which is paginated at 23 records, so it requires many, many round trips. Then many, many more round trips to retrieve the order history for each AccountId (which is also paginated). As noted above, this data comes back in different formats depending on the origin of the trade order (i.e. original site, pro, advanced trade, etc.)

My guess why Coinbase has all these AccountIds is to separate Wallets for payments/payouts. This might be beneficial in some outlying use cases where "BTC" payments should be separated for different products/services. But in their haste to add such feature creep, they neglected to support simple queries like all orders, all "BTC-USD" trades, or even all "BTC" accounts.

Sifting through this mess was both time-consuming and frustrating. None of this history/change is well documented. Their support is worse than non-existent.

Still, I like the Coinbase Exchange, and have used it for USD purchases of coins for over a decade. So I will be supporting their latest Advanced Trade API - for what it's worth.

Changes:

Gone are the V1 and Pro support. Forget about FIX. Support only latest Advanced Trade API (with silly Wallets) Sign all calls, as required. Correctly format trade histories. Support new pagination where required. Cache AccountIds locally to eliminate unnecessary round trips (this is problematic if new Ids are created/deleted, oh well..) Correctly fail when executed Public API calls that require an API Key which isn't set (this applies mostly to sockets).

I have most of this done, but there are a few APIs I need to test that I don't use in my own software. I'll update when this is completed.

O-Mutt commented 7 months ago

I have most of this done, but there are a few APIs I need to test that I don't use in my own software. I'll update when this is completed.

Is there somewhere that you are "keeping" this? Your fork has no active branches and it is not totally clear what this looks like for you.

BobDeCuir commented 7 months ago

O-Mutt,

Wow, I forgot I even forked this.

I have some unique requirements which causes me to keep the source local to my development environment. No worries, I have synced my fork and will update the Coinbase API this week and do a pull request. All is going well with the final Sockets and conversion to V3.

V3!

Yeah, I hope this is well received. After converting to V2, I found updated V3 docs on the Coinbase site. This new version is a little easier to work with, but like v2 will require signed headers on ALL calls (no more “Public”).

O-Mutt commented 7 months ago

O-Mutt,

Wow, I forgot I even forked this.

I have some unique requirements which causes me to keep the source local to my development environment. No worries, I have synced my fork and will update the Coinbase API this week and do a pull request. All is going well with the final Sockets and conversion to V3.

V3!

Yeah, I hope this is well received. After converting to V2, I found updated V3 docs on the Coinbase site. This new version is a little easier to work with, but like v2 will require signed headers on ALL calls (no more “Public”).

Any word here, friend?

BobDeCuir commented 7 months ago

@O-Mutt

Been working on it. I target C#12 locally so had to do some revision when compiling from the master branch. Most of the time has been spent switching between V2 and V3 and determining when to use where. (The changes in their APIs are all over the place with very little consistency.)

Status: Still finishing sockets. These required a near complete rewrite, and I didn’t want to deviate far from the base code. Should be completed by this weekend – but I’m leaving them blank in my fork until I’m satisfied. The PlaceOrder and Withdrawal calls have yet to be tested – so I’m also leaving them blank on my fork for now. I’m still not happy with the Transaction Parsing for legacy stuff, and am unsure if it is really needed. Seems adequate for now.

I’ve pushed the ExchangeCoinbaseAPI.cs file and it compiles if you want to take a look. I won’t be doing a pull request until everything is finished and fully tested. Hopefully by Monday.

Cheers.

O-Mutt commented 7 months ago

@O-Mutt

Been working on it. I target C#12 locally so had to do some revision when compiling from the master branch. Most of the time has been spent switching between V2 and V3 and determining when to use where. (The changes in their APIs are all over the place with very little consistency.)

Status: Still finishing sockets. These required a near complete rewrite, and I didn’t want to deviate far from the base code. Should be completed by this weekend – but I’m leaving them blank in my fork until I’m satisfied. The PlaceOrder and Withdrawal calls have yet to be tested – so I’m also leaving them blank on my fork for now. I’m still not happy with the Transaction Parsing for legacy stuff, and am unsure if it is really needed. Seems adequate for now.

I’ve pushed the ExchangeCoinbaseAPI.cs file and it compiles if you want to take a look. I won’t be doing a pull request until everything is finished and fully tested. Hopefully by Monday.

Cheers.

Solid check in. I have a team that is more than willing to assist if there is anything that we can help with. Fwiw I held off on moving forward with my branch for the time being so we didn't duplicate efforts

BobDeCuir commented 7 months ago

Create Pull Request on 12/03/1023.

Sorry, I didn't compile Tests or Console Apps.

Awaiting confirmation.