Coinbase Pro backtest not running

sparky2708 commented 2 years ago

Description

INFO: "enable_shorting" not specified in preferences, defaulting to: "True" Where do I specify it to be "false" or "true"?
KeyError: 'maker_fee_rate' API keys look right but not sure. Am seeing the following object for "fees" retrieved from CoinbasePro:

My Code:

coinbase = CoinbasePro()
s = Strategy(coinbase)
s.add_price_event(price_event, 'BTC-USD', resolution='30m', init=init)
result = s.backtest(initial_values={'USD': 10000}, to='2y')

settings.json

    "settings": {
      "account_update_time": 5000,
      "use_sandbox": false,
      "use_sandbox_websockets": false,
      "websocket_buffer_size": 10000,
      "test_connectivity_on_auth": true,

      "coinbase_pro": {
        "cash": "USD"
      },
      "binance": {
        "cash": "USDT",
        "binance_tld": "us"
      },
      "alpaca": {
        "websocket_stream": "iex",
        "cash": "USD"
      },
      "oanda": {
        "cash": "USD"
      }
    }
  }

backtest.json (if applicable)

backtest.json here

Error (if applicable)

Traceback (most recent call last):
  File "H:\ProgramData\Anaconda3\lib\runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "H:\ProgramData\Anaconda3\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "c:\Users\xxxxxxxxxx\.vscode\extensions\ms-python.python-2021.12.1559732655\pythonFiles\lib\python\debugpy\__main__.py", line 45, in <module>
    cli.main()
  File "c:\Users\xxxxxxxxxx\.vscode\extensions\ms-python.python-2021.12.1559732655\pythonFiles\lib\python\debugpy/..\debugpy\server\cli.py", line 444, in main
    run()
  File "c:\Users\xxxxxxxxxx\.vscode\extensions\ms-python.python-2021.12.1559732655\pythonFiles\lib\python\debugpy/..\debugpy\server\cli.py", line 285, in run_file
    runpy.run_path(target_as_str, run_name=compat.force_str("__main__"))
  File "H:\ProgramData\Anaconda3\lib\runpy.py", line 265, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "H:\ProgramData\Anaconda3\lib\runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "H:\ProgramData\Anaconda3\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "c:\git\crypto-test\src\macd\macd.py", line 57, in <module>
    s = Strategy(coinbase)
  File "C:\Users\xxxxxxxxxx\AppData\Roaming\Python\Python38\site-packages\blankly\frameworks\strategy\strategy_base.py", line 64, in __init__
    self.__paper_trade_exchange = blankly.PaperTrade(self.__exchange)
  File "C:\Users\xxxxxxxxxx\AppData\Roaming\Python\Python38\site-packages\blankly\exchanges\interfaces\paper_trade\paper_trade.py", line 28, in __init__
    self.interface = PaperTradeInterface(authenticated_exchange.get_interface(),
  File "C:\Users\xxxxxxxxxx\AppData\Roaming\Python\Python38\site-packages\blankly\exchanges\interfaces\paper_trade\paper_trade_interface.py", line 56, in __init__
    ExchangeInterface.__init__(self, derived_interface.get_exchange_type(), derived_interface)
  File "C:\Users\xxxxxxxxxx\AppData\Roaming\Python\Python38\site-packages\blankly\exchanges\interfaces\exchange_interface.py", line 135, in __init__
    self.init_exchange()
  File "C:\Users\xxxxxxxxxx\AppData\Roaming\Python\Python38\site-packages\blankly\exchanges\interfaces\paper_trade\paper_trade_interface.py", line 131, in init_exchange
    "maker_fee_rate": fees['maker_fee_rate'],
KeyError: 'maker_fee_rate'

Platform Info

Python version: Python 3.8.8 64-bit ("base", conda)
Platform: Windows 10

Additional context Add any other context about the problem here.

EmersonDove commented 2 years ago

Hi thanks for posting an issue!

Enable shorting is a new setting we added, take a look at this settings.json. The enable_shorting setting is under alpaca so it has no bearing on Coinbase Pro regardless of set true or false.
The coinbase timestamp message is new to me - a search brings up this thread that talks about the timestamp expired issue here. Your computer may just be sending an offset timestamp.

I might go ahead and add an exception for that query so that we can forward the Coinbase Pro error more easily but I will keep looking around to see if I can replicate the timing issue.

sparky2708 commented 2 years ago

Thank you for the lightning fast reply! Really love Blankly!

I suppose maybe it shouldn't print out as an info when using Coinbase
You are absolutely right, during one of the Windows Updates somehow the "Auto-Set Time" in Settings got turned off and my time drifted by about 3 minutes. Turned it on and it seems the code has moved passed it. Maybe a clearer error message would have helped here for anyone that might run into it later on.

sparky2708 commented 2 years ago

So just trying the sample MACD strategy. My code looks like:

coinbase = CoinbasePro()
s = Strategy(coinbase)
s.add_price_event(price_event, 'BTC-USD', resolution='30m', init=init)
result = s.backtest(initial_values={'USD': 10000}, to='2y')
print(result)

I get:

INFO: No portfolio name to load specified, defaulting to the first in the file: (my cool portfolio). This is fine if there is only one portfolio in use.
INFO: "enable_shorting" not specified in preferences, defaulting to: "True"
No cached data found for BTC-USD from: 1578075925 to 1641146125 at a resolution of 1800 seconds.
INFO: Granularity is not an accepted granularity...rounding to nearest valid value.
Progress: [##########] 99.75% 
Initializing...
Traceback (most recent call last):
  File "H:\ProgramData\Anaconda3\lib\runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "H:\ProgramData\Anaconda3\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "c:\Users\XXXXXXXX\.vscode\extensions\ms-python.python-2021.12.1559732655\pythonFiles\lib\python\debugpy\__main__.py", line 45, in <module>
    cli.main()
  File "c:\Users\XXXXXXXX\.vscode\extensions\ms-python.python-2021.12.1559732655\pythonFiles\lib\python\debugpy/..\debugpy\server\cli.py", line 444, in main
    run()
  File "c:\Users\XXXXXXXX\.vscode\extensions\ms-python.python-2021.12.1559732655\pythonFiles\lib\python\debugpy/..\debugpy\server\cli.py", line 285, in run_file
    runpy.run_path(target_as_str, run_name=compat.force_str("__main__"))
  File "H:\ProgramData\Anaconda3\lib\runpy.py", line 265, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "H:\ProgramData\Anaconda3\lib\runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "H:\ProgramData\Anaconda3\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "c:\git\crypto-test\src\macd\macd.py", line 59, in <module>
    result = s.backtest(initial_values={'USD': 10000}, to='2y', settings_path="e:\\git\\crypto-trading\\backtest.json")
  File "C:\Users\XXXXXXXX\AppData\Roaming\Python\Python38\site-packages\blankly\frameworks\strategy\strategy_base.py", line 517, in backtest
    results = self.backtesting_controller.run()
  File "C:\Users\XXXXXXXX\AppData\Roaming\Python\Python38\site-packages\blankly\exchanges\interfaces\paper_trade\backtest_controller.py", line 602, in run
    i['init'](i['asset_id'], i['state_object'])
  File "c:\git\crypto-test\src\macd\macd.py", line 17, in init
    variables['history'] = interface.history(symbol, 800,
  File "C:\Users\XXXXXXXX\AppData\Roaming\Python\Python38\site-packages\blankly\exchanges\interfaces\exchange_interface.py", line 212, in history
    valid_time_in_past = utils.ceil_date(parsed_date,
  File "C:\Users\XXXXXXXX\AppData\Roaming\Python\Python38\site-packages\blankly\utils\utils.py", line 605, in ceil_date
    return dt.fromtimestamp(date.timestamp() + secs - date.timestamp() % secs)
AttributeError: 'numpy.int32' object has no attribute 'timestamp'

EmersonDove commented 2 years ago

Thank you for the lightning fast reply! Really love Blankly!

I suppose maybe it shouldn't print out as an info when using Coinbase

You are absolutely right, during one of the Windows Updates somehow the "Auto-Set Time" in Settings got turned off and my time drifted by about 3 minutes. Turned it on and it seems the code has moved passed it. Maybe a clearer error message would have helped here for anyone that might run into it later on.

Awesome,

For us the settings are all blended together so I can't check if I should show an error per exchange but I can show where the setting should be set. Maybe the error should be enable_shorting not set in alpaca - defaulting to True so that users can localize it and know it's not attached to their exchange.

It looks like we need to wrap all coinbase responses and check for the message tag and throw errors when that occurs.

EmersonDove commented 2 years ago

For your second comment above this is a critical issue that we have to fix. For some reason on Windows systems pandas pushes datatypes to numpy which messes up time conversion. I just got free so I might just go fix this right now.

EmersonDove commented 2 years ago

Just pushed out a new release. If you update with pip install blankly --upgrade it should have a fix for that numpy issue.

sparky2708 commented 2 years ago

Great! Thank you. Will give it a shot tonight/tomorrow.

bfan1256 commented 2 years ago

Hey @sparky2708 , any updates on if this issue was fixed?

sparky2708 commented 2 years ago

Sorry, was out-of-town @bfan1256. Didn't work fully but got further in. Just trying to run your RSI script.

Progress: [##########] 99.99% H:\ProgramData\Anaconda3\lib\site-packages\blankly\metrics\portfolio.py:55: RuntimeWarning: invalid value encountered in double_scalars return mean / std Traceback (most recent call last): File "H:\ProgramData\Anaconda3\lib\runpy.py", line 194, in _run_module_as_main return _run_code(code, main_globals, None, File "H:\ProgramData\Anaconda3\lib\runpy.py", line 87, in _run_code exec(code, run_globals) File "c:\Users\me.vscode\extensions\ms-python.python-2021.12.1559732655\pythonFiles\lib\python\debugpy__main.py", line 45, in cli.main() File "c:\Users\me.vscode\extensions\ms-python.python-2021.12.1559732655\pythonFiles\lib\python\debugpy/..\debugpy\server\cli.py", line 444, in main run() File "c:\Users\me.vscode\extensions\ms-python.python-2021.12.1559732655\pythonFiles\lib\python\debugpy/..\debugpy\server\cli.py", line 285, in run_file runpy.run_path(target_as_str, run_name=compat.force_str("main__")) File "H:\ProgramData\Anaconda3\lib\runpy.py", line 265, in run_path return _run_module_code(code, init_globals, run_name, File "H:\ProgramData\Anaconda3\lib\runpy.py", line 97, in _run_module_code _run_code(code, mod_globals, init_globals, File "H:\ProgramData\Anaconda3\lib\runpy.py", line 87, in _run_code exec(code, run_globals) File "E:\git\crypto-trading/src/rsi/rsi.py", line 41, in main() File "E:\git\crypto-trading/src/rsi/rsi.py", line 37, in main backtest_result = s.backtest(initial_values={'USD': 10000}, to='1y', settings_path="backtest.json") File "H:\ProgramData\Anaconda3\lib\site-packages\blankly\frameworks\strategy\strategy_base.py", line 539, in backtest results = self.backtesting_controller.run() File "H:\ProgramData\Anaconda3\lib\site-packages\blankly\exchanges\interfaces\paper_trade\backtest_controller.py", line 910, in run if np.isnan(metrics_indicators[i]): TypeError: ufunc 'isnan' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''

EmersonDove commented 2 years ago

Hey, I believe this should be addressed in a recent update. Try running pip install blankly --upgrade and tell me how it goes when you get a chance.

bfan1256 commented 2 years ago

Hey @sparky2708 , let us know if this has fixed it! Cheers :D

blankly-finance / blankly