Discontinued support for CCXT

[cjdsellers]

After much thought and consideration it has been decided that continuing support for the CCXT library is not aligned with the best interests of the project for various reasons which I will elaborate on below.

The CCXT adapter has now been removed from the develop branch, and because of this will not be included in the upcoming 1.129.0 release.

We acknowledge that this is fairly short notice to discontinue support for what is currently one of the main integrations for the project, and arguably the most popular and used. Users are reminded that it's possible to continue using CCXT for trading if they pin their NautilusTrader version to 1.128.0.

Reasons for removal:

• The requirement for a paid subscription to ccxtpro for the integration to actually work. This is due to the streaming WebSocket endpoint support only being available in the paid for version. This has caused some confusion for users as to why its necessary to purchase an external subscription license for the platform to work for crypto trading. Additional confusion stems from the free open-source ccxt library vs the proprietary ccxtpro library.
• Recent breaking changes between the latest version of ccxt and older versions of ccxtpro has meant that core developers can't continue to run live examples and check the health of the integration.
• Work is about to commence on integrations for both Interactive Brokers and Binance, in tandem. The reason for this is to help ensure good design decisions are being made to support all future integrations. Continuing to support CCXT whilst fairly aggressive refactorings are taking place would have been an additional burden on development, slowing down progress.
• Possible confusion between the NautilusTrader Binance integration, and the CCXT-Binance integration. Meaning CCXT was slated for removal anyway once the first crypto exchange integration was completed, this has been brought forward.
• The platform provides some fairly advanced features and order types, the CCXT unified API does not support a lot of these and so even if CCXT was 100% integrated it would still mean a more limited NautilusTrader feature set is being provided for the supported exchanges.

I'd like to take the opportunity to thank the community for their patience through what has so far been some very aggressive refactorings resulting in continual breaking changes to the core API, and this sudden removal of CCXT only adds to the turbulence. As has been expressed in release notes, the purpose of this is to ensure the best possible foundation to build this platform and project on for years to come.

We will ensure that the integrations are prioritized so that traders who have invested into learning and developing on this platform are not 'left in the lurch'.

Again, I really appreciate your patience, and thank you so much for your continued support! Onwards! :rocket:

[OnlyC]

@yohplala Cryptofeed only subscribe data. They don't provide positions & orders REST requests.. I'm also using cryptofeed to get data for backtesting. Websocket endpoints work great, except the Open Interest that have to query through http, got errors from binance constantly. Maybe because I'm subscribing 9 pairs concurrently.

[yohplala]

@yohplala Cryptofeed only subscribe data. They don't provide positions & orders REST requests.. I'm also using cryptofeed to get data for backtesting. Websocket endpoints work great, except the Open Interest that have to query through http, got errors from binance constantly. Maybe because I'm subscribing 9 pairs concurrently.

Hi @OnlyC , my bad, I am baffled. My understanding that the rest folder was containing what was needed to manage orders. With the on-going work, the rest folder has been removed, and from what I can see, there is no utils indeed to actually set orders. Info about orders can be retrieved for some exchanges, but indeed, this is not enough to actually trade. https://github.com/bmoscon/cryptofeed/blob/master/docs/auth_channels.md I am very sorry for the useless noise.

[OnlyC]

@yohplala Yeah, REST was used to get data which isn't available from websockets, ex: Open Interest from Binance. I actually like the combo cryptofeed + free CCXT library but since there're issuescjdsellers listed, we should look for more extendable solution. Just need more time because we will have to write every exchange integration our self.

[cjdsellers]

I've personally found both cryptofeed and cryptostore to be fine solutions for capturing crypto market data.

However, based on previous experience with third party libraries, when it comes to integrations into a complex system such as this, it will probably pay off in the long run to write our own library. There are less surprises, and doing so offers more control to tailor to our exact requirements.

[thematz1]

python-binance library seems to be pretty solid in the realm of managing the websocket connections and having a fully featured API. I hear what you are saying about writing a library for nautilus in specific, I just wanted to point out the library incase you hadn't come across it. It is not an official API but has a lot of contributions @cjdsellers

https://github.com/sammchardy/python-binance

The library also has websocket support:

https://python-binance.readthedocs.io/en/latest/

[cjdsellers]

Thanks for the heads up on the library, I've browsed the source a few times already. We're also looking at https://github.com/binance/binance-connector-python as a reference too.

We do want to avoid dependencies under the hood onto heavy-weight frameworks like autobahn and twisted. Given these Binance libraries are MIT licensed though, we can take a lot of inspiration with acknowledge where appropriate.

One important attribute of the current architecture is the efficient use of a single event loop, implemented with uvloop (calls into the libuv C library under the hood). Introducing autobahn and twisted could hurt this in a few ways.

Edit: Above comments on dependencies mainly relate to the connector from Binance. I haven't looked at python-binance for a while and I see that its still very active. I'll look more closely, it could be a good option where alot of work has already been done.