I got an error with DRLEnsembleAgent.run_ensemble_strategy(). It misses timesteps_dict but it is in the code

[manu1dcb]

Describe the bug I am working on this notebook and, when I run this code df_summary = ensemble_agent.run_ensemble_strategy(A2C_model_kwargs, PPO_model_kwargs, DDPG_model_kwargs, TD3_model_kwargs, timesteps_dict) It gives me this error

**TypeError:** DRLEnsembleAgent.run_ensemble_strategy() missing 1 required positional argument: 'timesteps_dict' Does anybody know how can I fix it?

[RGIIST]

replace previous cell's content by

A2C_model_kwargs = {
                    'n_steps': 5,
                    'ent_coef': 0.005,
                    'learning_rate': 0.0007
                    }

PPO_model_kwargs = {
                    "ent_coef":0.01,
                    "n_steps": 2048,
                    "learning_rate": 0.00025,
                    "batch_size": 128
                    }

DDPG_model_kwargs = {
                      #"action_noise":"ornstein_uhlenbeck",
                      "buffer_size": 10_000,
                      "learning_rate": 0.0005,
                      "batch_size": 64
                    }

SAC_model_kwargs = {
    "batch_size": 64,
    "buffer_size": 100000,
    "learning_rate": 0.0001,
    "learning_starts": 100,
    "ent_coef": "auto_0.1",
}

TD3_model_kwargs = {"batch_size": 100, "buffer_size": 1000000, "learning_rate": 0.0001}

timesteps_dict = {'a2c' : 10_000,
                 'ppo' : 10_000,
                 'ddpg' : 10_000,
                 'sac' : 10_000,
                 'td3' : 10_000
                 }

It means you need to add SAC and TD3 kwargs. and then run

df_summary = ensemble_agent.run_ensemble_strategy(A2C_model_kwargs,
                                                 PPO_model_kwargs,
                                                 DDPG_model_kwargs,
                                                 SAC_model_kwargs,
                                                 TD3_model_kwargs,
                                                 timesteps_dict)

[manu1dcb]

replace previous cell's content by

A2C_model_kwargs = {
                    'n_steps': 5,
                    'ent_coef': 0.005,
                    'learning_rate': 0.0007
                    }

PPO_model_kwargs = {
                    "ent_coef":0.01,
                    "n_steps": 2048,
                    "learning_rate": 0.00025,
                    "batch_size": 128
                    }

DDPG_model_kwargs = {
                      #"action_noise":"ornstein_uhlenbeck",
                      "buffer_size": 10_000,
                      "learning_rate": 0.0005,
                      "batch_size": 64
                    }

SAC_model_kwargs = {
    "batch_size": 64,
    "buffer_size": 100000,
    "learning_rate": 0.0001,
    "learning_starts": 100,
    "ent_coef": "auto_0.1",
}

TD3_model_kwargs = {"batch_size": 100, "buffer_size": 1000000, "learning_rate": 0.0001}

timesteps_dict = {'a2c' : 10_000,
                 'ppo' : 10_000,
                 'ddpg' : 10_000,
                 'sac' : 10_000,
                 'td3' : 10_000
                 }

It means you need to add SAC and TD3 kwargs. and then run

df_summary = ensemble_agent.run_ensemble_strategy(A2C_model_kwargs,
                                                 PPO_model_kwargs,
                                                 DDPG_model_kwargs,
                                                 SAC_model_kwargs,
                                                 TD3_model_kwargs,
                                                 timesteps_dict)

Thanks Ravi! This gives me another issue:

`--------------------------------------------------------------------------- TypeError Traceback (most recent call last) Cell In[35], line 1 ----> 1 df_summary = ensemble_agent.run_ensemble_strategy(A2C_model_kwargs, 2 TD3_model_kwargs, 3 PPO_model_kwargs, 4 DDPG_model_kwargs, 5 SAC_model_kwargs, 6 timesteps_dict)

File ~\anaconda3\envs\RLFinance2\lib\site-packages\finrl\agents\stablebaselines3\models.py:577, in DRLEnsembleAgent.run_ensemble_strategy(self, A2C_model_kwargs, PPO_model_kwargs, DDPG_model_kwargs, SAC_model_kwargs, TD3_model_kwargs, timesteps_dict) 572 # print("training: ",len(data_split(df, start=20090000, end=test.datadate.unique()[i-rebalance_window]) )) 573 # print("==============Model Training===========") 574 # Train Each Model 575 for model_name in MODELS.keys(): 576 # Train The Model --> 577 model, sharpe_list, sharpe = self._train_window( 578 model_name, 579 kwargs[model_name], 580 model_dct[model_name]["sharpe_list"], 581 validation_start_date, 582 validation_end_date, 583 timesteps_dict, 584 i, 585 validation, 586 turbulence_threshold, 587 ) 588 # Save the model's sharpe ratios, and the model itself 589 model_dct[model_name]["sharpe_list"] = sharpe_list

File ~\anaconda3\envs\RLFinance2\lib\site-packages\finrl\agents\stablebaselines3\models.py:371, in DRLEnsembleAgent._train_window(self, model_name, model_kwargs, sharpe_list, validation_start_date, validation_end_date, timesteps_dict, i, validation, turbulence_threshold) 368 return None, sharpe_list, -1 370 print(f"======{model_name} Training========") --> 371 model = self.get_model( 372 model_name, self.train_env, policy="MlpPolicy", model_kwargs=model_kwargs 373 ) 374 model = self.train_model( 375 model, 376 model_name, (...) 379 total_timesteps=timesteps_dict[model_name], 380 ) # 100_000 381 print( 382 f"======{model_name} Validation from: ", 383 validation_start_date, 384 "to ", 385 validation_end_date, 386 )

File ~\anaconda3\envs\RLFinance2\lib\site-packages\finrl\agents\stablebaselines3\models.py:216, in DRLEnsembleAgent.get_model(model_name, env, policy, policy_kwargs, model_kwargs, seed, verbose) 212 temp_model_kwargs["action_noise"] = NOISE[ 213 temp_model_kwargs["action_noise"] 214 ](mean=np.zeros(n_actions), sigma=0.1 * np.ones(n_actions)) 215 print(temp_model_kwargs) --> 216 return MODELS[model_name]( 217 policy=policy, 218 env=env, 219 tensorboard_log=f"{config.TENSORBOARD_LOG_DIR}/{model_name}", 220 verbose=verbose, 221 policy_kwargs=policy_kwargs, 222 seed=seed, 223 **temp_model_kwargs, 224 )

TypeError: DDPG.init() got an unexpected keyword argument 'ent_coef' `

What's the problem? thanks!!

[RGIIST]

https://colab.research.google.com/drive/1h6ZDbUi5PgYHSGKJXg7djhKUMafFvHMG#scrollTo=tiFA3pRWPQMO&uniqifier=1

This is the link to my copy of the same Jupyter file. It is running smoothly and training is going on. Have a look at the code, this might help you.