nautechsystems / nautilus_trader

A high-performance algorithmic trading platform and event-driven backtester
https://nautilustrader.io
GNU Lesser General Public License v3.0
1.95k stars 445 forks source link

Authentication Error with CCXT-COINBASEPRO #308

Closed BlackWingedKing closed 3 years ago

BlackWingedKing commented 3 years ago

@cjdsellers I get this authentication error with CCXT-COINBASEPRO. Generated testnet keys from here

2021-05-06T01:22:06.318079Z [ERR] TESTER-001.ExecEngine: AuthenticationError(requires `password`)
 Traceback (most recent call last):
  File "nautilus_trader\\common\\component.pyx", line 312, in nautilus_trader.common.component.Component._trigger_fsm
    action()
  File "nautilus_trader\\execution\\engine.pyx", line 371, in nautilus_trader.execution.engine.ExecutionEngine._start
    cpdef void _start(self) except *:
  File "nautilus_trader\\execution\\engine.pyx", line 374, in nautilus_trader.execution.engine.ExecutionEngine._start
    client.connect()
  File "nautilus_trader\\adapters\\ccxt\\execution.pyx", line 147, in nautilus_trader.adapters.ccxt.execution.CCXTExecutionClient.connect
    if self._client.check_required_credentials():
  File "C:\Users\rites\.conda\envs\win_poetry\lib\site-packages\ccxt\base\exchange.py", line 1258, in check_required_credentials
    raise AuthenticationError('requires `' + key + '`')

I think this is because the COINBASEPRO api requires key, secret and password. but in the CCXTDataClientFactory we are only parsing apiKey and secret in internalConfig. So, may be you could add a check for coinbasepro and get the password in that case

ccxt's coinbasepro.py which requires password field

full output

2021-05-06T01:22:05.964058Z [INF] TESTER-001.ExecCache: Integrity check passed in 995μs.
2021-05-06T01:22:05.964058Z [INF] TESTER-001.ExecEngine: Loaded cache in 2037368μs.
2021-05-06T01:22:05.964058Z [INF] TESTER-001.ExecEngine: Registered <nautilus_trader.live.risk_engine.LiveRiskEngine object at 0x000001CD26B70F40>.
2021-05-06T01:22:05.965050Z [INF] TESTER-001.Trader: state=INITIALIZED...
2021-05-06T01:22:05.965050Z [INF] TESTER-001.Trader: Initializing strategies...
2021-05-06T01:22:05.966061Z [INF] TESTER-001.DataEngine: Registered EMACross(id=EMACross-001).
2021-05-06T01:22:05.966061Z [INF] TESTER-001.ExecEngine: Registered EMACross(id=EMACross-001).
2021-05-06T01:22:05.966061Z [INF] TESTER-001.EMACross-001: Set ClientOrderIdGenerator count to 0.
2021-05-06T01:22:05.967046Z [INF] TESTER-001.Trader: Initialized EMACross(id=EMACross-001).
2021-05-06T01:22:05.967046Z [INF] TESTER-001.EMACross-001: No user state to load.
2021-05-06T01:22:05.968059Z [INF] TESTER-001.TradingNode: state=INITIALIZED.
2021-05-06T01:22:05.968059Z [INF] TESTER-001.TradingNode: Initialized in 2.056s.
2021-05-06T01:22:06.310071Z [INF] TESTER-001.CCXTDataClient-COINBASE PRO: Initialized.
2021-05-06T01:22:06.311073Z [INF] TESTER-001.DataEngine: Registered CCXTDataClient-COINBASE PRO.
2021-05-06T01:22:06.315060Z [INF] TESTER-001.CCXTExecClient-COINBASE PRO: Initialized.
2021-05-06T01:22:06.315060Z [INF] TESTER-001.ExecEngine: Registered CCXTExecutionClient-COINBASE PRO.
2021-05-06T01:22:06.315060Z [INF] TESTER-001.RiskEngine: Registered CCXTExecutionClient-COINBASE PRO.
2021-05-06T01:22:06.316073Z [INF] TESTER-001.TradingNode: state=STARTING...
2021-05-06T01:22:06.316073Z [INF] TESTER-001.DataEngine: state=STARTING...
2021-05-06T01:22:06.316073Z [INF] TESTER-001.CCXTDataClient-COINBASE PRO: Connecting...
2021-05-06T01:22:06.316073Z [INF] TESTER-001.DataEngine: state=RUNNING.
2021-05-06T01:22:06.316073Z [INF] TESTER-001.ExecEngine: state=STARTING...
2021-05-06T01:22:06.316073Z [INF] TESTER-001.CCXTExecClient-COINBASE PRO: Connecting...
2021-05-06T01:22:06.318079Z [ERR] TESTER-001.ExecEngine: AuthenticationError(requires `password`)
 Traceback (most recent call last):
  File "nautilus_trader\\common\\component.pyx", line 312, in nautilus_trader.common.component.Component._trigger_fsm
    action()
  File "nautilus_trader\\execution\\engine.pyx", line 371, in nautilus_trader.execution.engine.ExecutionEngine._start
    cpdef void _start(self) except *:
  File "nautilus_trader\\execution\\engine.pyx", line 374, in nautilus_trader.execution.engine.ExecutionEngine._start
    client.connect()
  File "nautilus_trader\\adapters\\ccxt\\execution.pyx", line 147, in nautilus_trader.adapters.ccxt.execution.CCXTExecutionClient.connect
    if self._client.check_required_credentials():
  File "C:\Users\rites\.conda\envs\win_poetry\lib\site-packages\ccxt\base\exchange.py", line 1258, in check_required_credentials
    raise AuthenticationError('requires `' + key + '`')

2021-05-06T01:22:06.318079Z [INF] TESTER-001.ExecEngine: state=RUNNING.
Traceback (most recent call last):
  File "C:\Users\rites\Desktop\projects\nautilus_trader\examples\live\ccxt-coinbasepro_ema_cross.py", line 119, in <module>
    node.start()
  File "C:\Users\rites\Desktop\projects\nautilus_trader\nautilus_trader\live\node.py", line 358, in start
    self._loop.run_until_complete(self._run())
  File "C:\Users\rites\.conda\envs\win_poetry\lib\asyncio\base_events.py", line 642, in run_until_complete
    return future.result()
  File "C:\Users\rites\Desktop\projects\nautilus_trader\nautilus_trader\live\node.py", line 477, in _run
    self._exec_engine.start()
  File "nautilus_trader\\common\\component.pyx", line 180, in nautilus_trader.common.component.Component.start
    cpdef void start(self) except *:
  File "nautilus_trader\\common\\component.pyx", line 196, in nautilus_trader.common.component.Component.start
    self._trigger_fsm(
  File "nautilus_trader\\common\\component.pyx", line 315, in nautilus_trader.common.component.Component._trigger_fsm
    raise
  File "nautilus_trader\\common\\component.pyx", line 312, in nautilus_trader.common.component.Component._trigger_fsm
    action()
  File "nautilus_trader\\execution\\engine.pyx", line 371, in nautilus_trader.execution.engine.ExecutionEngine._start
    cpdef void _start(self) except *:
  File "nautilus_trader\\execution\\engine.pyx", line 374, in nautilus_trader.execution.engine.ExecutionEngine._start
    client.connect()
  File "nautilus_trader\\adapters\\ccxt\\execution.pyx", line 147, in nautilus_trader.adapters.ccxt.execution.CCXTExecutionClient.connect
    if self._client.check_required_credentials():
  File "C:\Users\rites\.conda\envs\win_poetry\lib\site-packages\ccxt\base\exchange.py", line 1258, in check_required_credentials
    raise AuthenticationError('requires `' + key + '`')
ccxt.base.errors.AuthenticationError: requires `password`

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\rites\Desktop\projects\nautilus_trader\examples\live\ccxt-coinbasepro_ema_cross.py", line 121, in <module>
    node.dispose()
  File "C:\Users\rites\Desktop\projects\nautilus_trader\nautilus_trader\live\node.py", line 402, in dispose
    self._data_engine.dispose()
  File "nautilus_trader\\common\\component.pyx", line 270, in nautilus_trader.common.component.Component.dispose
    cpdef void dispose(self) except *:
  File "nautilus_trader\\common\\component.pyx", line 289, in nautilus_trader.common.component.Component.dispose
    self._trigger_fsm(
  File "nautilus_trader\\common\\component.pyx", line 307, in nautilus_trader.common.component.Component._trigger_fsm
    raise  # Guards against component being put in an invalid state
  File "nautilus_trader\\common\\component.pyx", line 304, in nautilus_trader.common.component.Component._trigger_fsm
    self._fsm.trigger(trigger1)
  File "nautilus_trader\\core\\fsm.pyx", line 117, in nautilus_trader.core.fsm.FiniteStateMachine.trigger
    raise InvalidStateTrigger(f"{self.state_string_c()} -> {self._trigger_parser(trigger)}")
nautilus_trader.core.fsm.InvalidStateTrigger: RUNNING -> DISPOSE
Task was destroyed but it is pending!
task: <Task pending name='Task-2' coro=<LiveLogger._consume_messages() running at nautilus_trader\common\logging.pyx:654>>
2021-05-06T01:22:11.368695Z [WRN] TESTER-001.TradingNode: Timed out (5s) waiting for node to stop.
2021-05-06T01:22:11.368695Z [INF] TESTER-001.TradingNode: state=DISPOSING...
2021-05-06T01:22:11.368695Z [INF] TESTER-001.Trader: state=DISPOSING...
2021-05-06T01:22:11.368695Z [INF] TESTER-001.EMACross-001: state=DISPOSING...
2021-05-06T01:22:11.368695Z [INF] TESTER-001.EMACross-001: state=DISPOSED.
2021-05-06T01:22:11.368695Z [INF] TESTER-001.Trader: state=DISPOSED.
2021-05-06T01:22:11.370492Z [ERR] TESTER-001.DataEngine: InvalidStateTrigger(RUNNING -> DISPOSE)
 Traceback (most recent call last):
  File "C:\Users\rites\Desktop\projects\nautilus_trader\examples\live\ccxt-coinbasepro_ema_cross.py", line 119, in <module>
    node.start()
  File "C:\Users\rites\Desktop\projects\nautilus_trader\nautilus_trader\live\node.py", line 358, in start
    self._loop.run_until_complete(self._run())
  File "C:\Users\rites\.conda\envs\win_poetry\lib\asyncio\base_events.py", line 642, in run_until_complete
    return future.result()
  File "C:\Users\rites\Desktop\projects\nautilus_trader\nautilus_trader\live\node.py", line 477, in _run
    self._exec_engine.start()
  File "nautilus_trader\\common\\component.pyx", line 180, in nautilus_trader.common.component.Component.start
    cpdef void start(self) except *:
  File "nautilus_trader\\common\\component.pyx", line 196, in nautilus_trader.common.component.Component.start
    self._trigger_fsm(
  File "nautilus_trader\\common\\component.pyx", line 315, in nautilus_trader.common.component.Component._trigger_fsm
    raise
  File "nautilus_trader\\common\\component.pyx", line 312, in nautilus_trader.common.component.Component._trigger_fsm
    action()
  File "nautilus_trader\\execution\\engine.pyx", line 371, in nautilus_trader.execution.engine.ExecutionEngine._start
    cpdef void _start(self) except *:
  File "nautilus_trader\\execution\\engine.pyx", line 374, in nautilus_trader.execution.engine.ExecutionEngine._start
    client.connect()
  File "nautilus_trader\\adapters\\ccxt\\execution.pyx", line 147, in nautilus_trader.adapters.ccxt.execution.CCXTExecutionClient.connect
    if self._client.check_required_credentials():
  File "C:\Users\rites\.conda\envs\win_poetry\lib\site-packages\ccxt\base\exchange.py", line 1258, in check_required_credentials
    raise AuthenticationError('requires `' + key + '`')
ccxt.base.errors.AuthenticationError: requires `password`

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "nautilus_trader\\common\\component.pyx", line 304, in nautilus_trader.common.component.Component._trigger_fsm
    self._fsm.trigger(trigger1)
  File "nautilus_trader\\core\\fsm.pyx", line 117, in nautilus_trader.core.fsm.FiniteStateMachine.trigger
    raise InvalidStateTrigger(f"{self.state_string_c()} -> {self._trigger_parser(trigger)}")

2021-05-06T01:22:11.370492Z [INF] TESTER-001.TradingNode: Closing event loop...
2021-05-06T01:22:11.370492Z [INF] TESTER-001.TradingNode: loop.is_running=False
2021-05-06T01:22:11.370492Z [INF] TESTER-001.TradingNode: loop.is_closed=True
2021-05-06T01:22:11.370492Z [INF] TESTER-001.TradingNode: state=DISPOSED.
Task was destroyed but it is pending!
task: <Task pending name='Task-3' coro=<CCXTDataClient._run_after_delay() running at nautilus_trader\adapters\ccxt\data.pyx:959> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x000001CD274A1820>()]>>
Task was destroyed but it is pending!
task: <Task pending name='Task-6' coro=<LiveDataEngine._run_message_queue() running at nautilus_trader\live\data_engine.pyx:283> cb=[gather.<locals>._done_callback() at C:\Users\rites\.conda\envs\win_poetry\lib\asyncio\tasks.py:764]>
Task was destroyed but it is pending!
task: <Task pending name='Task-5' coro=<LiveDataEngine._run_data_queue() running at nautilus_trader\live\data_engine.pyx:267> cb=[gather.<locals>._done_callback() at C:\Users\rites\.conda\envs\win_poetry\lib\asyncio\tasks.py:764]>
sys:1: RuntimeWarning: coroutine 'CCXTDataClient._subscribed_instruments_update' was never awaited
Task was destroyed but it is pending!
task: <Task pending name='Task-7' coro=<Exchange.load_markets_helper() running at C:\Users\rites\.conda\envs\win_poetry\lib\site-packages\ccxt\async_support\base\exchange.py:168> cb=[<TaskWakeupMethWrapper object at 0x000001CD274A19A0>()]>
sys:1: RuntimeWarning: coroutine 'Exchange.load_markets_helper' was never awaited
Task was destroyed but it is pending!
task: <Task pending name='Task-4' coro=<CCXTDataClient._connect() running at nautilus_trader\adapters\ccxt\data.pyx:181> wait_for=<Task pending name='Task-7' coro=<Exchange.load_markets_helper() running at C:\Users\rites\.conda\envs\win_poetry\lib\site-packages\ccxt\async_support\base\exchange.py:168> cb=[<TaskWakeupMethWrapper object at 0x000001CD274A19A0>()]>>
BlackWingedKing commented 3 years ago

The code which I ran (modified from ccxt-binance_ema_cross.py for coinbasepro, didn't attach the imports)

# The configuration dictionary can come from anywhere such as a JSON or YAML
# file. Here it is hardcoded into the example for clarity.
config = {
    "trader": {
        "name": "TESTER",  # Not sent beyond system boundary
        "id_tag": "001",  # Used to ensure orders are unique for this trader
    },
    "system": {
        "loop_debug": False,  # The event loop debug mode
        "connection_timeout": 5.0,  # Timeout for successful connections for all engine clients
        "disconnection_timeout": 5.0,  # Timeout for successful disconnection for all engine clients
        "check_residuals_delay": 5.0,  # How long to wait after stopping for residual events (secs)
    },
    "logging": {
        "level_stdout": "INF",
    },
    "exec_database": {
        "type": "redis",
        "host": "localhost",
        "port": 6379,
    },
    "risk": {},
    "strategy": {
        "load_state": True,  # Strategy state is loaded from the database on start
        "save_state": True,  # Strategy state is saved to the database on shutdown
    },
    "data_clients": {
        "CCXT-COINBASEPRO": {
            "account_id": "CB_ACCOUNT_ID",  # value is the environment variable key
            "api_key": "CB_API_KEY",  # value is the environment variable key
            "api_secret": "CB_API_SECRET",  # value is the environment variable key
            "api_password": "CB_API_PASSWORD",  # value is the environment variable key            
            "sandbox_mode": True,  # If client uses the testnet
        },
    },
    "exec_clients": {
        "CCXT-COINBASEPRO": {
            "account_id": "CB_ACCOUNT_ID",  # value is the environment variable key
            "api_key": "CB_API_KEY",  # value is the environment variable key
            "api_secret": "CB_API_SECRET",  # value is the environment variable key
            "api_password": "CB_API_PASSWORD",  # value is the environment variable key
            "sandbox_mode": True,  # If client uses the testnet
        },
    },
}

# Instantiate your strategies to pass into the trading node. You could add
# custom options into the configuration file or even use another configuration
# file.

instrument_id = InstrumentId(
    symbol=Symbol("ETH/USDT"),
    venue=Venue("COINBASEPRO"),
)

strategy = EMACross(
    instrument_id=instrument_id,
    bar_spec=BarSpecification(1, BarAggregation.MINUTE, PriceType.LAST),
    fast_ema_period=10,
    slow_ema_period=20,
    trade_size=Decimal("0.05"),
    order_id_tag="001",
)

# Instantiate the node passing a list of strategies and configuration
node = TradingNode(strategies=[strategy], config=config)

# Register your client factories with the node (can take user defined factories)
node.add_data_client_factory("CCXT", CCXTDataClientFactory)
node.add_exec_client_factory("CCXT", CCXTExecutionClientFactory)
node.build()

# Stop and dispose of the node with SIGINT/CTRL+C
if __name__ == "__main__":
    try:
        node.start()
    finally:
        node.dispose()
cjdsellers commented 3 years ago

I'll look into this tomorrow.

In the mean time I've added the capability to cache and persist Currency objects so we shouldn't run into the issue again where a currency can't be de-serialized.

2021-05-06T12:28:13.344024Z [INF] TESTER-001.ExecCache: Cached 412 currencies from database.
2021-05-06T12:28:13.344489Z [INF] TESTER-001.ExecCache: Cached 1 account from database.
2021-05-06T12:28:13.344690Z [INF] TESTER-001.ExecCache: Cached 0 orders from database.
2021-05-06T12:28:13.344858Z [INF] TESTER-001.ExecCache: Cached 0 positions from database.
2021-05-06T12:28:13.344888Z [INF] TESTER-001.ExecCache: Checking data integrity...
cjdsellers commented 3 years ago

Upon reviewing their docs, it appears that some exchanges do require a password argument. https://github.com/ccxt/ccxt/wiki/Manual#authentication

I've added the following into the config parsing

        # Build internal configuration
        cdef dict internal_config = {
            "apiKey": os.getenv(config.get("api_key", ""), ""),
            "secret": os.getenv(config.get("api_secret", ""), ""),
            "password": os.getenv(config.get("api_password", ""), ""),

Which should sort the current authentication issue.

The change will make it to the develop branch on my next push.

BlackWingedKing commented 3 years ago

Thank you!