Error training the agent for Paper_Trading

[lcavalie]

When using FinRL_PaperTrading_Demo on a brand new installation (Python 3.9.13) the notebook crashes at the training step.

train(start_date = '2021-10-11', 
      end_date = '2021-10-15',
      ticker_list = ticker_list, 
      data_source = 'alpaca',
      time_interval= candle_time_interval, 
      technical_indicator_list= INDICATORS,
      drl_lib='elegantrl', 
      env=env,
      model_name='ppo', 
      API_KEY = API_KEY, 
      API_SECRET = API_SECRET, 
      API_BASE_URL = API_BASE_URL,
      erl_params=ERL_PARAMS,
      cwd='./papertrading_erl', #current_working_dir
      break_step=1e5)

the following error happens

Data clean finished!
Successfully transformed into array
| Arguments Remove cwd: ./papertrading_erl
################################################################################
ID     Step    maxR |    avgR   stdR   avgS  stdS |    expR   objC   etc.
---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
Cell In [19], line 1
----> 1 train(start_date = '2021-10-11', 
      2       end_date = '2021-10-15',
      3       ticker_list = ticker_list, 
      4       data_source = 'alpaca',
      5       time_interval= candle_time_interval, 
      6       technical_indicator_list= INDICATORS,
      7       drl_lib='elegantrl', 
      8       env=env,
      9       model_name='ppo', 
     10       API_KEY = API_KEY, 
     11       API_SECRET = API_SECRET, 
     12       API_BASE_URL = API_BASE_URL,
     13       erl_params=ERL_PARAMS,
     14       cwd='./papertrading_erl', #current_working_dir
     15       break_step=1e5)

File ~/miniforge3/envs/FinRL/lib/python3.9/site-packages/finrl/train.py:60, in train(start_date, end_date, ticker_list, data_source, time_interval, technical_indicator_list, drl_lib, env, model_name, if_vix, **kwargs)
     53     agent = DRLAgent_erl(
     54         env=env,
     55         price_array=price_array,
     56         tech_array=tech_array,
     57         turbulence_array=turbulence_array,
     58     )
     59     model = agent.get_model(model_name, model_kwargs=erl_params)
---> 60     trained_model = agent.train_model(
     61         model=model, cwd=cwd, total_timesteps=break_step
     62     )
     63 elif drl_lib == "rllib":
     64     total_episodes = kwargs.get("total_episodes", 100)

File ~/miniforge3/envs/FinRL/lib/python3.9/site-packages/finrl/agents/elegantrl/models.py:84, in DRLAgent.train_model(self, model, cwd, total_timesteps)
     82 model.cwd = cwd
     83 model.break_step = total_timesteps
---> 84 train_and_evaluate(model)

File ~/miniforge3/envs/FinRL/lib/python3.9/site-packages/elegantrl/train/run.py:115, in train_and_evaluate(***failed resolving arguments***)
    113     r_exp = trajectory[3].mean().item()
    114     torch.set_grad_enabled(True)
--> 115     logging_tuple = agent.update_net(trajectory)
    116     torch.set_grad_enabled(False)
    117 (if_reach_goal, if_save) = evaluator.evaluate_save_and_plot(agent.act, steps, r_exp, logging_tuple)

File ~/miniforge3/envs/FinRL/lib/python3.9/site-packages/elegantrl/agents/AgentPPO.py:159, in AgentPPO.update_net(self, buffer)
    156 obj_actors = 0.0
    158 update_times = int(buffer_size * self.repeat_times / self.batch_size)
--> 159 assert update_times >= 1
    160 for _ in range(update_times):
    161     indices = torch.randint(buffer_size, size=(self.batch_size,), requires_grad=False)

AssertionError: 

This happens without changing any parameter, just the plain notebook as downloaded from the site. Not sure if the error is in ElegantRL or int he parameters passed to train the agent. update_times is at 0 because buffer_size=32 self.repeat_times=1 and self.batch_size=2048 when the error happens

This can be reproduced by just running the notebook. Same thing happens on Mac M1 and on ubuntu (aarch64)

After further investigation the same issue is also present in other notebooks, for example FinRL_Compare_ElegantRL_RLlib_Stablebaseline3 exhibit the exact same error message, so I suspect it's related to ElegantRL somehow)

[ZiyiXia]

A new version of the paper trading notebook was added. Please check it out.

[lcavalie]

I still have to test this but it is very likely to fix the problem. The code of this notebook was completely off and significantly different from the similarly named notebook in FinRL-meta. Now the 2 are aligned similar. I found a bug in the FinRl-meta notebook that is corrected in this newly updated notebook in FinRL. I will report it in FinRL-meta later on after more testing. Thanks for the update.