Describe the bug
Modify the ticker_list to NAS_100_TICKER[:30] the program fails to load the model.
To Reproduce
Steps to reproduce the behavior:
Open examples/FinRL_PaperTrading_Demo_refactored.py
Scroll down to 'from finrl.config_tickers import DOW_30_TICKER'
Change this and next line to:
from finrl.config_tickers import DOW_30_TICKER, NAS_100_TICKER
ticker_list = NAS_100_TICKER[:30]
Also change the following occurences of DOW_30_TICKER:
-action_dim = len(DOW_30_TICKER)
+action_dim = len(ticker_list)
state_dim = (
1 + 2 + 3 action_dim + len(INDICATORS) action_dim
) # Calculate the DRL state dimension manually for paper trading. amount + (turbulence, turbulence_bool) + (price, shares, cd (holding time)) * stock_dim + tech_dim
paper_trading_erl = PaperTradingAlpaca(
ticker_list=DOW_30_TICKER,
ticker_list=ticker_list,
Run the program, you see that the test and training steps are running ok, but fails at the
TRAIN_START_DATE: 2024-02-05
TRAIN_END_DATE: 2024-02-12
TEST_START_DATE: 2024-02-13
TEST_END_DATE: 2024-02-14
TRAINFULL_START_DATE: 2024-02-05
TRAINFULL_END_DATE: 2024-02-14
Alpaca successfully connected
| step: Number of samples, or total training steps, or running times of env.step().
| time: Time spent from the start of training to this moment.
| avgR: Average value of cumulative rewards, which is the sum of rewards in an episode.
| stdR: Standard dev of cumulative rewards, which is the sum of rewards in an episode.
| avgS: Average of steps in an episode.
| objC: Objective of Critic network. Or call it loss function of critic network.
| objA: Objective of Actor network. It is the average Q value of the critic network.
| step time | avgR stdR avgS | objC objA
| 2.00e+04 14 | 4.11 0.00 2339 | 0.21 0.37
| 4.00e+04 26 | 4.15 0.00 2339 | 0.18 0.38
| 6.00e+04 38 | 5.01 0.00 2339 | 0.17 0.36
| 8.00e+04 49 | 3.77 0.00 2339 | 0.14 0.36
| 1.00e+05 61 | 3.06 0.00 2339 | 0.15 0.37
Alpaca successfully connected
price_array: 780
| load actor from: ./papertrading_erl/actor.pth
/Users/mingqxu/opt/anaconda3/envs/finrl/lib/python3.8/site-packages/finrl/meta/paper_trading/common.py:660: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at /Users/runner/work/pytorch/pytorch/pytorch/torch/csrc/utils/tensor_new.cpp:233.)
s_tensor = _torch.as_tensor((state,), device=device)
Test Finished!
episode_return 1.0080942088954623
Alpaca successfully connected
| step: Number of samples, or total training steps, or running times of env.step().
| time: Time spent from the start of training to this moment.
| avgR: Average value of cumulative rewards, which is the sum of rewards in an episode.
| stdR: Standard dev of cumulative rewards, which is the sum of rewards in an episode.
| avgS: Average of steps in an episode.
| objC: Objective of Critic network. Or call it loss function of critic network.
| objA: Objective of Actor network. It is the average Q value of the critic network.
| step time | avgR stdR avgS | objC objA
| 2.00e+04 12 | -2.27 0.00 3119 | 0.21 0.37
| 4.00e+04 24 | -0.20 0.00 3119 | 0.16 0.36
| 6.00e+04 36 | 0.02 0.00 3119 | 0.20 0.36
| 8.00e+04 48 | -0.59 0.00 3119 | 0.14 0.38
| 1.00e+05 60 | -0.24 0.00 3119 | 0.17 0.36
| 1.20e+05 72 | 0.31 0.00 3119 | 0.19 0.36
| 1.40e+05 84 | -0.86 0.00 3119 | 0.18 0.38
| 1.60e+05 96 | 0.64 0.00 3119 | 0.21 0.39
| 1.80e+05 108 | 0.86 0.00 3119 | 0.12 0.37
| 2.00e+05 119 | 0.92 0.00 3119 | 0.18 0.37
| load actor from: ./papertrading_erl_retrain/actor.pth
Traceback (most recent call last):
File "/Users/mingqxu/opt/anaconda3/envs/finrl/lib/python3.8/site-packages/finrl/meta/paper_trading/alpaca.py", line 49, in init
actor.load_state_dict(
File "/Users/mingqxu/opt/anaconda3/envs/finrl/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1671, in load_state_dict
raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for ActorPPO:
size mismatch for action_std_log: copying a param with shape torch.Size([1, 26]) from checkpoint, the shape in current model is torch.Size([1, 30]).
size mismatch for net.0.weight: copying a param with shape torch.Size([128, 289]) from checkpoint, the shape in current model is torch.Size([128, 333]).
size mismatch for net.4.weight: copying a param with shape torch.Size([26, 64]) from checkpoint, the shape in current model is torch.Size([30, 64]).
size mismatch for net.4.bias: copying a param with shape torch.Size([26]) from checkpoint, the shape in current model is torch.Size([30]).
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "FinRL_PaperTrading_Demo_refactored.py", line 146, in
paper_trading_erl = PaperTradingAlpaca(
File "/Users/mingqxu/opt/anaconda3/envs/finrl/lib/python3.8/site-packages/finrl/meta/paper_trading/alpaca.py", line 55, in init
raise ValueError("Fail to load agent!")
ValueError: Fail to load agent!
Expected behavior
The code should load the model and continue to trade.
Screenshots
N/A
Desktop (please complete the following information):
Describe the bug Modify the ticker_list to NAS_100_TICKER[:30] the program fails to load the model.
To Reproduce Steps to reproduce the behavior:
ticker_list = NAS_100_TICKER[:30]
Also change the following occurences of DOW_30_TICKER:
-action_dim = len(DOW_30_TICKER) +action_dim = len(ticker_list) state_dim = ( 1 + 2 + 3 action_dim + len(INDICATORS) action_dim ) # Calculate the DRL state dimension manually for paper trading. amount + (turbulence, turbulence_bool) + (price, shares, cd (holding time)) * stock_dim + tech_dim
paper_trading_erl = PaperTradingAlpaca(
|
step: Number of samples, or total training steps, or running times ofenv.step(). |time: Time spent from the start of training to this moment. |avgR: Average value of cumulative rewards, which is the sum of rewards in an episode. |stdR: Standard dev of cumulative rewards, which is the sum of rewards in an episode. |avgS: Average of steps in an episode. |objC: Objective of Critic network. Or call it loss function of critic network. |objA: Objective of Actor network. It is the average Q value of the critic network. | step time | avgR stdR avgS | objC objA | 2.00e+04 14 | 4.11 0.00 2339 | 0.21 0.37 | 4.00e+04 26 | 4.15 0.00 2339 | 0.18 0.38 | 6.00e+04 38 | 5.01 0.00 2339 | 0.17 0.36 | 8.00e+04 49 | 3.77 0.00 2339 | 0.14 0.36 | 1.00e+05 61 | 3.06 0.00 2339 | 0.15 0.37 Alpaca successfully connected price_array: 780 | load actor from: ./papertrading_erl/actor.pth /Users/mingqxu/opt/anaconda3/envs/finrl/lib/python3.8/site-packages/finrl/meta/paper_trading/common.py:660: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at /Users/runner/work/pytorch/pytorch/pytorch/torch/csrc/utils/tensor_new.cpp:233.) s_tensor = _torch.as_tensor((state,), device=device) Test Finished! episode_return 1.0080942088954623 Alpaca successfully connected|
step: Number of samples, or total training steps, or running times ofenv.step(). |time: Time spent from the start of training to this moment. |avgR: Average value of cumulative rewards, which is the sum of rewards in an episode. |stdR: Standard dev of cumulative rewards, which is the sum of rewards in an episode. |avgS: Average of steps in an episode. |objC: Objective of Critic network. Or call it loss function of critic network. |objA: Objective of Actor network. It is the average Q value of the critic network. | step time | avgR stdR avgS | objC objA | 2.00e+04 12 | -2.27 0.00 3119 | 0.21 0.37 | 4.00e+04 24 | -0.20 0.00 3119 | 0.16 0.36 | 6.00e+04 36 | 0.02 0.00 3119 | 0.20 0.36 | 8.00e+04 48 | -0.59 0.00 3119 | 0.14 0.38 | 1.00e+05 60 | -0.24 0.00 3119 | 0.17 0.36 | 1.20e+05 72 | 0.31 0.00 3119 | 0.19 0.36 | 1.40e+05 84 | -0.86 0.00 3119 | 0.18 0.38 | 1.60e+05 96 | 0.64 0.00 3119 | 0.21 0.39 | 1.80e+05 108 | 0.86 0.00 3119 | 0.12 0.37 | 2.00e+05 119 | 0.92 0.00 3119 | 0.18 0.37 | load actor from: ./papertrading_erl_retrain/actor.pth Traceback (most recent call last): File "/Users/mingqxu/opt/anaconda3/envs/finrl/lib/python3.8/site-packages/finrl/meta/paper_trading/alpaca.py", line 49, in init actor.load_state_dict( File "/Users/mingqxu/opt/anaconda3/envs/finrl/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1671, in load_state_dict raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format( RuntimeError: Error(s) in loading state_dict for ActorPPO: size mismatch for action_std_log: copying a param with shape torch.Size([1, 26]) from checkpoint, the shape in current model is torch.Size([1, 30]). size mismatch for net.0.weight: copying a param with shape torch.Size([128, 289]) from checkpoint, the shape in current model is torch.Size([128, 333]). size mismatch for net.4.weight: copying a param with shape torch.Size([26, 64]) from checkpoint, the shape in current model is torch.Size([30, 64]). size mismatch for net.4.bias: copying a param with shape torch.Size([26]) from checkpoint, the shape in current model is torch.Size([30]).During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "FinRL_PaperTrading_Demo_refactored.py", line 146, in
paper_trading_erl = PaperTradingAlpaca(
File "/Users/mingqxu/opt/anaconda3/envs/finrl/lib/python3.8/site-packages/finrl/meta/paper_trading/alpaca.py", line 55, in init
raise ValueError("Fail to load agent!")
ValueError: Fail to load agent!
Expected behavior The code should load the model and continue to trade.
Screenshots N/A
Desktop (please complete the following information):
Additional context finrl 0.3.5
Package Version
absl-py 1.4.0 aiodns 3.0.0 aiohttp 3.9.3 aiohttp-cors 0.7.0 aiosignal 1.3.1 alpaca-py 0.15.0 alpaca-trade-api 3.2.0 annotated-types 0.6.0 anyio 3.6.2 appdirs 1.4.4 appnope 0.1.3 argon2-cffi 21.3.0 argon2-cffi-bindings 21.2.0 arrow 1.2.3 asttokens 2.2.1 astunparse 1.6.3 async-timeout 4.0.2 attrs 22.2.0 backcall 0.2.0 beautifulsoup4 4.11.2 bleach 6.0.0 blessed 1.20.0 box2d-py 2.3.5 cachetools 5.3.0 ccxt 2.8.75 certifi 2022.12.7 cffi 1.15.1 charset-normalizer 2.1.1 click 8.1.3 cloudpickle 2.2.1 colorful 0.5.5 comm 0.1.2 contourpy 1.0.7 cryptography 39.0.1 cvxpy 1.3.0 cycler 0.11.0 debugpy 1.6.6 decorator 5.1.1 defusedxml 0.7.1 deprecation 2.1.0 distlib 0.3.6 ecos 2.0.12 elegantrl 0.3.6 empyrical 0.5.5 entrypoints 0.4 exchange-calendars 3.6.3 executing 1.2.0 fastjsonschema 2.16.3 filelock 3.9.0 finrl 0.3.5 flatbuffers 23.1.21 fonttools 4.38.0 fqdn 1.5.1 frozendict 2.3.5 frozenlist 1.3.3 gast 0.4.0 google-api-core 2.11.0 google-auth 2.16.1 google-auth-oauthlib 0.4.6 google-pasta 0.2.0 googleapis-common-protos 1.58.0 gpustat 1.0.0 GPUtil 1.4.0 greenlet 2.0.2 grpcio 1.49.1 gym 0.21.0 h5py 3.8.0 html5lib 1.1 idna 3.4 importlib-metadata 4.13.0 importlib-resources 5.12.0 ipykernel 6.21.2 ipython 8.11.0 ipython-genutils 0.2.0 ipywidgets 8.0.4 isoduration 20.11.0 jedi 0.18.2 Jinja2 3.1.2 joblib 1.2.0 jqdatasdk 1.8.11 jsonpointer 2.3 jsonschema 4.17.3 jupyter 1.0.0 jupyter_client 8.0.3 jupyter-console 6.6.2 jupyter_core 5.2.0 jupyter-events 0.6.3 jupyter_server 2.3.0 jupyter_server_terminals 0.4.4 jupyterlab-pygments 0.2.2 jupyterlab-widgets 3.0.5 keras 2.11.0 kiwisolver 1.4.4 korean-lunar-calendar 0.3.1 libclang 15.0.6.1 lxml 4.9.2 lz4 4.3.2 Markdown 3.4.1 MarkupSafe 2.1.2 matplotlib 3.7.0 matplotlib-inline 0.1.6 meta 1.0.2 mistune 2.0.5 msgpack 1.0.3 multidict 6.0.4 multitasking 0.0.11 nbclassic 0.5.2 nbclient 0.7.2 nbconvert 7.2.9 nbformat 5.7.3 nest-asyncio 1.5.6 notebook 6.5.2 notebook_shim 0.2.2 numpy 1.24.2 nvidia-ml-py 11.495.46 oauthlib 3.2.2 opencensus 0.11.1 opencensus-context 0.1.3 opt-einsum 3.3.0 osqp 0.6.2.post8 packaging 23.0 pandas 1.5.3 pandas-datareader 0.10.0 pandocfilters 1.5.0 parso 0.8.3 pexpect 4.8.0 pickleshare 0.7.5 Pillow 9.4.0 pip 22.3.1 pkgutil_resolve_name 1.3.10 platformdirs 3.0.0 ply 3.11 prometheus-client 0.16.0 prompt-toolkit 3.0.38 protobuf 3.19.6 psutil 5.9.4 psycopg2-binary 2.9.5 ptyprocess 0.7.0 pure-eval 0.2.2 py-spy 0.3.14 pyasn1 0.4.8 pyasn1-modules 0.2.8 pycares 4.3.0 pycparser 2.21 pydantic 2.5.2 pydantic_core 2.14.5 pyfolio 0.9.2+75.g4b901f6 pyglet 2.0.5 Pygments 2.14.0 pyluach 2.2.0 PyMySQL 1.0.2 pyparsing 3.0.9 pyrsistent 0.19.3 python-dateutil 2.8.2 python-json-logger 2.0.7 pytz 2022.7.1 PyYAML 6.0.1 pyzmq 25.0.0 qdldl 0.1.5.post3 qtconsole 5.4.0 QtPy 2.3.0 ray 2.3.0 requests 2.31.0 requests-oauthlib 1.3.1 rfc3339-validator 0.1.4 rfc3986-validator 0.1.1 rsa 4.9 scikit-learn 1.2.1 scipy 1.10.1 scs 3.2.2 seaborn 0.12.2 Send2Trash 1.8.0 setuptools 64.0.2 six 1.16.0 smart-open 6.3.0 sniffio 1.3.0 soupsieve 2.4 SQLAlchemy 1.4.46 sseclient-py 1.8.0 stable-baselines3 1.7.0 stack-data 0.6.2 stockstats 0.5.2 swig 4.1.1 tabulate 0.9.0 tensorboard 2.11.2 tensorboard-data-server 0.6.1 tensorboard-plugin-wit 1.8.1 tensorboardX 2.6 tensorflow 2.11.0 tensorflow-estimator 2.11.0 tensorflow-io-gcs-filesystem 0.31.0 termcolor 2.2.0 terminado 0.17.1 threadpoolctl 3.1.0 thriftpy2 0.4.16 tinycss2 1.2.1 toolz 0.12.0 torch 1.13.1 tornado 6.2 traitlets 5.9.0 typing_extensions 4.9.0 uri-template 1.2.0 urllib3 1.26.14 virtualenv 20.20.0 wcwidth 0.2.6 webcolors 1.12 webencodings 0.5.1 websocket-client 1.5.1 websockets 12.0 Werkzeug 2.2.3 wheel 0.38.4 widgetsnbextension 4.0.5 wrapt 1.15.0 wrds 3.1.5 yarl 1.8.2 yfinance 0.2.12 zipp 3.15.0