nautechsystems / nautilus_trader

A high-performance algorithmic trading platform and event-driven backtester
https://nautilustrader.io
GNU Lesser General Public License v3.0
1.95k stars 445 forks source link

Error with ccxt-coinbasepro testnet #311

Closed BlackWingedKing closed 3 years ago

BlackWingedKing commented 3 years ago

@cjdsellers I get this error with CCXT-COINBASEPRO. Generated testnet keys from here

2021-05-08T08:47:45.815754Z [INF] TESTER-001.ExecCache: Cached 8 currencies from database.
2021-05-08T08:47:45.818742Z [INF] TESTER-001.ExecCache: Cached 1 account from database.
2021-05-08T08:47:45.819746Z [INF] TESTER-001.ExecCache: Cached 1 order from database.
2021-05-08T08:47:45.820741Z [INF] TESTER-001.ExecCache: Cached 0 positions from database.
2021-05-08T08:47:45.821744Z [INF] TESTER-001.ExecCache: Checking data integrity...
2021-05-08T08:47:45.821744Z [INF] TESTER-001.ExecCache: Integrity check passed in 0μs.
2021-05-08T08:47:45.821744Z [INF] TESTER-001.ExecEngine: Loaded cache in 2030508μs.
2021-05-08T08:47:45.822747Z [INF] TESTER-001.ExecEngine: Registered <nautilus_trader.live.risk_engine.LiveRiskEngine object at 0x000001F957CD2720>.
2021-05-08T08:47:45.822747Z [INF] TESTER-001.Trader: state=INITIALIZED...
2021-05-08T08:47:45.823741Z [INF] TESTER-001.Trader: Initializing strategies...
2021-05-08T08:47:45.823741Z [INF] TESTER-001.DataEngine: Registered EMACross(id=EMACross-001).
2021-05-08T08:47:45.824744Z [INF] TESTER-001.ExecEngine: Registered EMACross(id=EMACross-001).
2021-05-08T08:47:45.824744Z [INF] TESTER-001.EMACross-001: Set ClientOrderIdGenerator count to 1.
2021-05-08T08:47:45.824744Z [INF] TESTER-001.Trader: Initialized EMACross(id=EMACross-001).
2021-05-08T08:47:45.825749Z [INF] TESTER-001.EMACross-001: No user state to load.
2021-05-08T08:47:45.825749Z [INF] TESTER-001.TradingNode: state=INITIALIZED.
2021-05-08T08:47:45.826747Z [INF] TESTER-001.TradingNode: Initialized in 2.053s.
2021-05-08T08:47:46.278310Z [INF] TESTER-001.CCXTDataClient-COINBASE PRO: Initialized.
2021-05-08T08:47:46.279309Z [INF] TESTER-001.DataEngine: Registered CCXTDataClient-COINBASE PRO.
2021-05-08T08:47:46.284310Z [INF] TESTER-001.CCXTExecClient-COINBASE PRO: Initialized.
2021-05-08T08:47:46.284310Z [INF] TESTER-001.ExecEngine: Registered CCXTExecutionClient-COINBASE PRO.
2021-05-08T08:47:46.285316Z [INF] TESTER-001.RiskEngine: Registered CCXTExecutionClient-COINBASE PRO.
2021-05-08T08:47:46.285316Z [INF] TESTER-001.TradingNode: state=STARTING...
2021-05-08T08:47:46.285316Z [INF] TESTER-001.DataEngine: state=STARTING...
2021-05-08T08:47:46.285316Z [INF] TESTER-001.CCXTDataClient-COINBASE PRO: Connecting...
2021-05-08T08:47:46.285316Z [INF] TESTER-001.DataEngine: state=RUNNING.
2021-05-08T08:47:46.285316Z [INF] TESTER-001.ExecEngine: state=STARTING...
2021-05-08T08:47:46.285316Z [INF] TESTER-001.CCXTExecClient-COINBASE PRO: Connecting...
2021-05-08T08:47:46.285316Z [INF] TESTER-001.CCXTExecClient-COINBASE PRO: API credentials validated.
2021-05-08T08:47:46.286319Z [INF] TESTER-001.Portfolio: Initialized 0 working orders.
2021-05-08T08:47:46.286319Z [INF] TESTER-001.Portfolio: Initialized 0 open positions.
2021-05-08T08:47:46.286319Z [INF] TESTER-001.Portfolio: Initialized 0 closed positions.
2021-05-08T08:47:46.286319Z [INF] TESTER-001.ExecEngine: state=RUNNING.
2021-05-08T08:47:46.286319Z [INF] TESTER-001.RiskEngine: state=STARTING...
2021-05-08T08:47:46.286319Z [INF] TESTER-001.RiskEngine: state=RUNNING.
2021-05-08T08:47:46.286319Z [INF] TESTER-001.TradingNode: Waiting for engines to initialize (5.0s timeout)...
Task exception was never retrieved
future: <Task finished name='Task-8' coro=<<coroutine without __name__>()> exception=ValueError("The 'lot_size' was not a positive real, was 0.0")>
Traceback (most recent call last):
  File "nautilus_trader\\adapters\\ccxt\\execution.pyx", line 163, in _connect
    await self._load_instruments()
  File "nautilus_trader\\adapters\\ccxt\\execution.pyx", line 437, in _load_instruments
    await self._instrument_provider.load_all_async()
  File "nautilus_trader\\adapters\\ccxt\\providers.pyx", line 68, in load_all_async
    self._load_instruments()
  File "nautilus_trader\\adapters\\ccxt\\providers.pyx", line 85, in nautilus_trader.adapters.ccxt.providers.CCXTInstrumentProvider._load_instruments
    instrument = self._parse_instrument(instrument_id, v)
  File "nautilus_trader\\adapters\\ccxt\\providers.pyx", line 210, in nautilus_trader.adapters.ccxt.providers.CCXTInstrumentProvider._parse_instrument
    return Instrument(
  File "nautilus_trader\\model\\instrument.pyx", line 149, in nautilus_trader.model.instrument.Instrument.__init__
    Condition.positive(lot_size, "lot_size")
  File "nautilus_trader\\core\\correctness.pyx", line 597, in nautilus_trader.core.correctness.Condition.positive
    raise make_exception(
ValueError: The 'lot_size' was not a positive real, was 0.0
Task exception was never retrieved
future: <Task finished name='Task-4' coro=<<coroutine without __name__>()> exception=ValueError("The 'lot_size' was not a positive real, was 0.0")>
Traceback (most recent call last):
  File "nautilus_trader\\adapters\\ccxt\\data.pyx", line 183, in _connect
    await self._load_instruments()
  File "nautilus_trader\\adapters\\ccxt\\data.pyx", line 964, in _load_instruments
    await self._instrument_provider.load_all_async()
  File "nautilus_trader\\adapters\\ccxt\\providers.pyx", line 68, in load_all_async
    self._load_instruments()
  File "nautilus_trader\\adapters\\ccxt\\providers.pyx", line 85, in nautilus_trader.adapters.ccxt.providers.CCXTInstrumentProvider._load_instruments
    instrument = self._parse_instrument(instrument_id, v)
  File "nautilus_trader\\adapters\\ccxt\\providers.pyx", line 210, in nautilus_trader.adapters.ccxt.providers.CCXTInstrumentProvider._parse_instrument
    return Instrument(
  File "nautilus_trader\\model\\instrument.pyx", line 149, in nautilus_trader.model.instrument.Instrument.__init__
    Condition.positive(lot_size, "lot_size")
  File "nautilus_trader\\core\\correctness.pyx", line 597, in nautilus_trader.core.correctness.Condition.positive
    raise make_exception(
ValueError: The 'lot_size' was not a positive real, was 0.0
2021-05-08T08:47:51.286338Z [ERR] TESTER-001.TradingNode: Timed out (5.0s) waiting for engines to connect.
Traceback (most recent call last):
  File "C:\Users\rites\Desktop\projects\eruditis\win_poetry\nautilus_test\examples\live\ccxt-coinbasepro_ema_cross.py", line 121, in <module>
    node.dispose()
  File "C:\Users\rites\Desktop\projects\eruditis\win_poetry\nautilus_test\nautilus_trader\live\node.py", line 402, in dispose
    self._data_engine.dispose()
  File "nautilus_trader\\common\\component.pyx", line 270, in nautilus_trader.common.component.Component.dispose
    cpdef void dispose(self) except *:
  File "nautilus_trader\\common\\component.pyx", line 289, in nautilus_trader.common.component.Component.dispose
    self._trigger_fsm(
  File "nautilus_trader\\common\\component.pyx", line 307, in nautilus_trader.common.component.Component._trigger_fsm
    raise  # Guards against component being put in an invalid state
  File "nautilus_trader\\common\\component.pyx", line 304, in nautilus_trader.common.component.Component._trigger_fsm
    self._fsm.trigger(trigger1)
  File "nautilus_trader\\core\\fsm.pyx", line 117, in nautilus_trader.core.fsm.FiniteStateMachine.trigger
    raise InvalidStateTrigger(f"{self.state_string_c()} -> {self._trigger_parser(trigger)}")
nautilus_trader.core.fsm.InvalidStateTrigger: RUNNING -> DISPOSE
BlackWingedKing commented 3 years ago

The code which I ran (modified from ccxt-binance_ema_cross.py for coinbasepro, didn't attach the imports)

# The configuration dictionary can come from anywhere such as a JSON or YAML
# file. Here it is hardcoded into the example for clarity.
config = {
    "trader": {
        "name": "TESTER",  # Not sent beyond system boundary
        "id_tag": "001",  # Used to ensure orders are unique for this trader
    },
    "system": {
        "loop_debug": False,  # The event loop debug mode
        "connection_timeout": 5.0,  # Timeout for successful connections for all engine clients
        "disconnection_timeout": 5.0,  # Timeout for successful disconnection for all engine clients
        "check_residuals_delay": 5.0,  # How long to wait after stopping for residual events (secs)
    },
    "logging": {
        "level_stdout": "INF",
    },
    "exec_database": {
        "type": "redis",
        "host": "localhost",
        "port": 6379,
    },
    "risk": {},
    "strategy": {
        "load_state": True,  # Strategy state is loaded from the database on start
        "save_state": True,  # Strategy state is saved to the database on shutdown
    },
    "data_clients": {
        "CCXT-COINBASEPRO": {
            "account_id": "CB_ACCOUNT_ID",  # value is the environment variable key
            "api_key": "CB_API_KEY",  # value is the environment variable key
            "api_secret": "CB_API_SECRET",  # value is the environment variable key
            "api_password": "CB_API_PASSWORD",  # value is the environment variable key            
            "sandbox_mode": True,  # If client uses the testnet
        },
    },
    "exec_clients": {
        "CCXT-COINBASEPRO": {
            "account_id": "CB_ACCOUNT_ID",  # value is the environment variable key
            "api_key": "CB_API_KEY",  # value is the environment variable key
            "api_secret": "CB_API_SECRET",  # value is the environment variable key
            "api_password": "CB_API_PASSWORD",  # value is the environment variable key
            "sandbox_mode": True,  # If client uses the testnet
        },
    },
}

# Instantiate your strategies to pass into the trading node. You could add
# custom options into the configuration file or even use another configuration
# file.

instrument_id = InstrumentId(
    symbol=Symbol("ETH/USDT"),
    venue=Venue("COINBASEPRO"),
)

strategy = EMACross(
    instrument_id=instrument_id,
    bar_spec=BarSpecification(1, BarAggregation.MINUTE, PriceType.LAST),
    fast_ema_period=10,
    slow_ema_period=20,
    trade_size=Decimal("0.05"),
    order_id_tag="001",
)

# Instantiate the node passing a list of strategies and configuration
node = TradingNode(strategies=[strategy], config=config)

# Register your client factories with the node (can take user defined factories)
node.add_data_client_factory("CCXT", CCXTDataClientFactory)
node.add_exec_client_factory("CCXT", CCXTExecutionClientFactory)
node.build()

# Stop and dispose of the node with SIGINT/CTRL+C
if __name__ == "__main__":
    try:
        node.start()
    finally:
        node.dispose()
cjdsellers commented 3 years ago

Looks like the CCXTInstrumentProvider is parsing out a lot size of 0 and triggering the following in Instrument

Condition.positive(lot_size, "lot_size")

If you comment out line 149 in model/instrument.pyx that should fix the issue as long as you don't need to reference a lot_size of 0.

The instruments are about to receive a heavy refactoring to create instruments per asset class and I'll be mindful of this error.

cjdsellers commented 3 years ago

I've come up with something better ahead of the 1.118.0 release.

        lot_size = values["info"].get("lotSize")
        if lot_size is not None and Decimal(lot_size) > 0:
            lot_size = Quantity(lot_size)
        else:
            lot_size = Quantity(1)

This ensures the lot_size is never 0.

Its definitely not the most efficient solution with the creation of the temporary decimal, however it'll work for now until the Instrument classes are revisited - likely including the creation of a CryptoInstrument.