search

robcarver17 / pysystemtrade

Systematic Trading in python
GNU General Public License v3.0
2.57k stars 816 forks source link

AttributeError for scripts run_backups and run_reports #320

Closed faiz31 closed 3 years ago

faiz31 commented 3 years ago

$SCRIPT_PATH/run_backups

Multiple and adjusted succeeded, then received AttributeError. Shortened output:

2021-03-28:2129.24 {'type': '', 'component': 'csvFuturesMultiplePricesData', 'instrument_code': 'WHEAT'}  Written multiple prices for WHEAT to /home/faiz/backups_csv/multiple_prices/WHEAT.csv
2021-03-28:2129.25 {'type': '', 'component': 'csvFuturesMultiplePricesData', 'instrument_code': 'WHEAT'}  Added data for instrument WHEAT
2021-03-28:2129.25 {'type': ''}  Written .csv backup multiple prices for WHEAT
No update needed
No update needed
No update needed
No update needed
No update needed
No update needed
2021-03-28:2129.26 {'type': '', 'component': 'csvFuturesAdjustedPricesData', 'instrument_code': 'COPPER'}  Added data for instrument COPPER
2021-03-28:2129.27 {'type': ''}  Written .csv backup for adjusted prices COPPER
2021-03-28:2129.27 {'type': '', 'component': 'csvFuturesAdjustedPricesData', 'instrument_code': 'CORN'}  Added data for instrument CORN
2021-03-28:2129.27 {'type': ''}  Written .csv backup for adjusted prices CORN
2021-03-28:2129.28 {'type': '', 'component': 'csvFuturesAdjustedPricesData', 'instrument_code': 'CRUDE_W'}  Added data for instrument CRUDE_W
2021-03-28:2129.28 {'type': ''}  Written .csv backup for adjusted prices CRUDE_W
2021-03-28:2129.28 {'type': '', 'component': 'csvFuturesAdjustedPricesData', 'instrument_code': 'EDOLLAR'}  Added data for instrument EDOLLAR
2021-03-28:2129.29 {'type': ''}  Written .csv backup for adjusted prices EDOLLAR
No update needed
No update needed
No update needed
No update needed
2021-03-28:2129.30 {'type': '', 'component': 'csvFuturesAdjustedPricesData', 'instrument_code': 'GOLD'}  Added data for instrument GOLD
2021-03-28:2129.31 {'type': ''}  Written .csv backup for adjusted prices GOLD
No update needed
No update needed
No update needed
No update needed
2021-03-28:2129.32 {'type': '', 'component': 'csvFuturesAdjustedPricesData', 'instrument_code': 'LEANHOG'}  Added data for instrument LEANHOG
2021-03-28:2129.32 {'type': ''}  Written .csv backup for adjusted prices LEANHOG
2021-03-28:2129.33 {'type': '', 'component': 'csvFuturesAdjustedPricesData', 'instrument_code': 'LIVECOW'}  Added data for instrument LIVECOW
2021-03-28:2129.33 {'type': ''}  Written .csv backup for adjusted prices LIVECOW
No update needed
No update needed
No update needed
No update needed
2021-03-28:2129.34 {'type': '', 'component': 'csvFuturesAdjustedPricesData', 'instrument_code': 'PALLAD'}  Added data for instrument PALLAD
2021-03-28:2129.34 {'type': ''}  Written .csv backup for adjusted prices PALLAD
2021-03-28:2129.35 {'type': '', 'component': 'csvFuturesAdjustedPricesData', 'instrument_code': 'PLAT'}  Added data for instrument PLAT
2021-03-28:2129.35 {'type': ''}  Written .csv backup for adjusted prices PLAT
No update needed
2021-03-28:2129.35 {'type': '', 'component': 'csvFuturesAdjustedPricesData', 'instrument_code': 'SOYBEAN'}  Added data for instrument SOYBEAN
2021-03-28:2129.36 {'type': ''}  Written .csv backup for adjusted prices SOYBEAN
No update needed
2021-03-28:2129.36 {'type': '', 'component': 'csvFuturesAdjustedPricesData', 'instrument_code': 'US10'}  Added data for instrument US10
2021-03-28:2129.37 {'type': ''}  Written .csv backup for adjusted prices US10
2021-03-28:2129.37 {'type': '', 'component': 'csvFuturesAdjustedPricesData', 'instrument_code': 'US2'}  Added data for instrument US2
2021-03-28:2129.38 {'type': ''}  Written .csv backup for adjusted prices US2
2021-03-28:2129.38 {'type': '', 'component': 'csvFuturesAdjustedPricesData', 'instrument_code': 'US20'}  Added data for instrument US20
2021-03-28:2129.38 {'type': ''}  Written .csv backup for adjusted prices US20
2021-03-28:2129.39 {'type': '', 'component': 'csvFuturesAdjustedPricesData', 'instrument_code': 'US5'}  Added data for instrument US5
2021-03-28:2129.39 {'type': ''}  Written .csv backup for adjusted prices US5
2021-03-28:2129.39 {'type': '', 'component': 'csvFuturesAdjustedPricesData', 'instrument_code': 'V2X'}  Added data for instrument V2X
2021-03-28:2129.40 {'type': ''}  Written .csv backup for adjusted prices V2X
No update needed
2021-03-28:2129.40 {'type': '', 'component': 'csvFuturesAdjustedPricesData', 'instrument_code': 'WHEAT'}  Added data for instrument WHEAT
2021-03-28:2129.41 {'type': ''}  Written .csv backup for adjusted prices WHEAT
Traceback (most recent call last):
  File "/home/faiz/pysystemtrade/sysproduction/linux/scripts/run.py", line 63, in <module>
    func(*args, **kwargs)
  File "/home/faiz/pysystemtrade/sysproduction/run_backups.py", line 14, in run_backups
    backup_process.run_process()
  File "/home/faiz/pysystemtrade/syscontrol/run_process.py", line 104, in run_process
    self._main_loop_over_methods()
  File "/home/faiz/pysystemtrade/syscontrol/run_process.py", line 126, in _main_loop_over_methods
    timer_class.check_and_run()
  File "/home/faiz/pysystemtrade/syscontrol/timer_functions.py", line 121, in check_and_run
    self.run_function()
  File "/home/faiz/pysystemtrade/syscontrol/timer_functions.py", line 299, in run_function
    self._function()
  File "/home/faiz/pysystemtrade/sysproduction/backup_arctic_to_csv.py", line 63, in backup_arctic_to_csv
    backup_strategy_position_data(backup_data)
  File "/home/faiz/pysystemtrade/sysproduction/backup_arctic_to_csv.py", line 265, in backup_strategy_position_data
    strategy_list = get_list_of_strategies(data)
  File "/home/faiz/pysystemtrade/sysproduction/data/strategies.py", line 45, in get_list_of_strategies
    return get_list_of_strategies_from_config(data)
  File "/home/faiz/pysystemtrade/sysproduction/data/strategies.py", line 55, in get_list_of_strategies_from_config
    list_of_strategies = diag_strategies_config.get_list_of_strategies()
  File "/home/faiz/pysystemtrade/sysproduction/data/strategies.py", line 26, in get_list_of_strategies
    list_of_strategies = list(strategy_dict.keys())
AttributeError: 'named_object' object has no attribute 'keys'

Same error for run reports:

$SCRIPT_PATH/run_reports

sysproduction.run_reports.run_reports:
None

Arguments:
[]

2021-03-28:1733.30 {'type': ''}  roll_report will run every 60 minutes at most 1 times with heartbeats every 10 minutes
2021-03-28:1733.30 {'type': ''}  daily_pandl_report will run every 60 minutes at most 1 times with heartbeats every 10 minutes
2021-03-28:1733.30 {'type': ''}  reconcile_report will run every 60 minutes at most 1 times with heartbeats every 10 minutes
2021-03-28:1733.31 {'type': ''}  trade_report will run every 60 minutes at most 1 times with heartbeats every 10 minutes
2021-03-28:1733.31 {'type': ''}  strategy_report will run every 60 minutes at most 1 times with heartbeats every 10 minutes
2021-03-28:1733.32 {'type': ''}  risk_report will run every 60 minutes at most 1 times with heartbeats every 10 minutes
2021-03-28:1733.32 {'type': ''}  status_report will run every 60 minutes at most 1 times with heartbeats every 10 minutes
2021-03-28:1733.33 {'type': 'run_reports'}  roll_report still alive, done 0 of 1 executions every 60 minutes
2021-03-28:1733.33 {'type': ''}  Running report Roll report sysproduction.reporting.roll_report.roll_info email {'instrument_code': 'ALL'}
2021-03-28:1734.35 {'type': ''}  Sent email subject Roll report
2021-03-28:1734.36 {'type': 'run_reports'}  roll_report executed 1 times so done
2021-03-28:1734.36 {'type': 'run_reports'}  daily_pandl_report still alive, done 0 of 1 executions every 60 minutes
2021-03-28:1734.36 {'type': ''}  Running report One day P&L report sysproduction.reporting.pandl_report.pandl_info email {'calendar_days_back': 1}
Getting p&l for KR3
KR3
20210600
Getting p&l for GBP
GBP
20210600
Getting p&l for EDOLLAR
EDOLLAR
20230900
Getting p&l for GOLD
GOLD
20210400
Getting p&l for US2
US2
20210600
Getting p&l for JPY
JPY
20210600
Getting p&l for KR10
KR10
20210600
Getting p&l for CORN
CORN
20211200
Getting p&l for OAT
OAT
20210600
Getting p&l for CRUDE_W
CRUDE_W
20221200
Getting p&l for EUR
EUR
20210600
Getting p&l for US20
US20
20210600
Getting p&l for SOYBEAN
SOYBEAN
20211100
Getting p&l for US5
US5
20210600
Getting p&l for V2X
V2X
20210400
Getting p&l for BOBL
BOBL
20210600
Getting p&l for MXP
MXP
20210600
Getting p&l for GAS_US
GAS_US
20210400
Getting p&l for BTP
BTP
20210600
Getting p&l for WHEAT
WHEAT
20211200
Getting p&l for example
Data for example CRUDE_W
Data for example EDOLLAR
Data for example EUR
Data for example GAS_US
Data for example GBP
Data for example GOLD
Data for example JPY
Data for example MXP
Data for example US2
Data for example US20
Data for example US5
Data for example BOBL
Data for example OAT
Data for example V2X
Data for example CORN
Data for example KR10
Data for example KR3
Data for example SOYBEAN
Data for example WHEAT
Data for example BTP
Getting data for Vol
Getting p&l for V2X
Getting p&l for VIX
Getting data for Metals
Getting p&l for COPPER
Getting p&l for GOLD
Getting p&l for PALLAD
Getting p&l for PLAT
Getting data for FX
Getting p&l for AUD
Getting p&l for EUR
Getting p&l for GBP
Getting p&l for JPY
Getting p&l for MXP
Getting p&l for NZD
Getting data for Equity
Getting p&l for AEX
Getting p&l for CAC
Getting p&l for EUROSTX
Getting p&l for KOSPI
Getting p&l for NASDAQ
Getting p&l for SMI
Getting p&l for SP500
Getting data for Bond
Getting p&l for BOBL
Getting p&l for BTP
Getting p&l for BUND
Getting p&l for EDOLLAR
Getting p&l for KR10
Getting p&l for KR3
Getting p&l for OAT
Getting p&l for SHATZ
Getting p&l for US10
Getting p&l for US2
Getting p&l for US20
Getting p&l for US5
Getting data for Ags
Getting p&l for CORN
Getting p&l for LEANHOG
Getting p&l for LIVECOW
Getting p&l for SOYBEAN
Getting p&l for WHEAT
Getting data for OilGas
Getting p&l for CRUDE_W
Getting p&l for GAS_US
2021-03-28:1734.56 {'type': ''}  Sent email subject One day P&L report
2021-03-28:1734.57 {'type': 'run_reports'}  daily_pandl_report executed 1 times so done
2021-03-28:1734.57 {'type': 'run_reports'}  reconcile_report still alive, done 0 of 1 executions every 60 minutes
2021-03-28:1734.57 {'type': ''}  Running report Reconcile report sysproduction.reporting.reconcile_report.reconcile_info email {}
2021-03-28:1734.58 {'type': '', 'component': 'mongoIbBrokerClientIdData'}  Locked IB client ID 10
2021-03-28:1735.11 {'type': '', 'broker': 'IB', 'clientid': 10}  Sent email subject Reconcile report
2021-03-28:1735.11 {'type': 'run_reports', 'broker': 'IB', 'clientid': 10}  reconcile_report executed 1 times so done
2021-03-28:1735.12 {'type': 'run_reports'}  trade_report still alive, done 0 of 1 executions every 60 minutes
2021-03-28:1735.12 {'type': ''}  Running report Trade report sysproduction.reporting.trades_report.trades_info email {'calendar_days_back': 1}
2021-03-28:1735.14 {'type': ''}  Sent email subject Trade report
2021-03-28:1735.15 {'type': 'run_reports'}  trade_report executed 1 times so done
2021-03-28:1735.15 {'type': 'run_reports'}  strategy_report still alive, done 0 of 1 executions every 60 minutes
2021-03-28:1735.16 {'type': ''}  Running report Strategy report sysproduction.reporting.strategies_report.strategy_report email {}
Traceback (most recent call last):
  File "/home/faiz/pysystemtrade/sysproduction/linux/scripts/run.py", line 63, in <module>
    func(*args, **kwargs)
  File "/home/faiz/pysystemtrade/sysproduction/run_reports.py", line 16, in run_reports
    price_process.run_process()
  File "/home/faiz/pysystemtrade/syscontrol/run_process.py", line 104, in run_process
    self._main_loop_over_methods()
  File "/home/faiz/pysystemtrade/syscontrol/run_process.py", line 126, in _main_loop_over_methods
    timer_class.check_and_run()
  File "/home/faiz/pysystemtrade/syscontrol/timer_functions.py", line 121, in check_and_run
    self.run_function()
  File "/home/faiz/pysystemtrade/syscontrol/timer_functions.py", line 299, in run_function
    self._function()
  File "/home/faiz/pysystemtrade/sysproduction/run_reports.py", line 48, in email_trades_report
    run_report(self.config, data=self.data)
  File "/home/faiz/pysystemtrade/sysproduction/reporting/reporting_functions.py", line 26, in run_report
    run_report_with_data_blob(report_config, data)
  File "/home/faiz/pysystemtrade/sysproduction/reporting/reporting_functions.py", line 41, in run_report_with_data_blob
    report_results = report_function(data, **report_kwargs)
  File "/home/faiz/pysystemtrade/sysproduction/reporting/strategies_report.py", line 27, in strategy_report
    list_of_strategies = get_list_of_strategies(data=data)
  File "/home/faiz/pysystemtrade/sysproduction/data/strategies.py", line 45, in get_list_of_strategies
    return get_list_of_strategies_from_config(data)
  File "/home/faiz/pysystemtrade/sysproduction/data/strategies.py", line 55, in get_list_of_strategies_from_config
    list_of_strategies = diag_strategies_config.get_list_of_strategies()
  File "/home/faiz/pysystemtrade/sysproduction/data/strategies.py", line 26, in get_list_of_strategies
    list_of_strategies = list(strategy_dict.keys())
AttributeError: 'named_object' object has no attribute 'keys'

Linux Mint

python 3.8.5 pandas 1.0.5 matplotlib 3.3.2 pyyaml 5.3.1 numpy 1.20.1 scipy 1.5.2 pymongo 3.9.0 arctic 1.79.2 ib-insync 0.9.65 psutil 5.6.6 pytest 6.2.2

robcarver17 commented 3 years ago

Can you paste a copy of your private yaml below (excluding any private details like account numbers etc)

faiz31 commented 3 years ago

That was it! My private config yaml was missing strategy configuration due to recent changes in defaults.yaml that I hadn't noticed. My strategy name is 'example', and adding the following, run backups and reports completed successfully:

## Strategy configuration
strategy_list:
  example:
    load_backtests:
      object: sysproduction.strategy_code.run_system_classic.runSystemClassic
      function: system_method
    reporting_code:
      function: sysproduction.strategy_code.report_system_classic.report_system_classic

# Allocate capital to strategies
# strategy_weights *must* be defined in private config
strategy_capital_allocation:
  function: sysproduction.strategy_code.strategy_allocation.weighted_strategy_allocation
  strategy_weights:
    example: 100.0