Unable to train on many agents

[hmomin]

Whenever I try to train on agents, I consistently get the error:

AttributeError: type object 'AgentXYZ' has no attribute 'if_off_policy'

For example, here is the error for AgentDQN:

Traceback (most recent call last):
  File "/home/momin/Documents/GitHub/ElegantRL/tests/test_training_agents.py", line 50, in test_should_create_arguments_for_each_agent
    Arguments(agent, env_func=gym.make, env_args=self.discrete_env_args)
  File "/home/momin/Documents/GitHub/ElegantRL/elegantrl/train/config.py", line 140, in __init__
    self.if_off_policy = agent.if_off_policy  # agent is on-policy or off-policy
AttributeError: type object 'AgentDQN' has no attribute 'if_off_policy'

I can confirm that this error affects the following agents: AgentDQN AgentD3QN AgentDDPG AgentDiscretePPO AgentDoubleDQN AgentDuelingDQN AgentModSAC AgentPPO_H AgentPPO AgentSAC_H AgentSAC AgentTD3

@shixun404

[hmomin]

Shixun, if you'd like me to help out with this, I'm happy to. However, since I've noticed you're working very closely with the agents files, I'd prefer you handle this to avoid any merge conflicts between us. I'm mostly fixing some issues that have popped up related to Isaac Gym.

[shixun404]

Sure, no problem at all. I will handle it this evening.

On Thu, Mar 3, 2022 at 4:59 PM Momin Haider @.***> wrote:

Shixun, if you'd like me to help out with this, I'm happy to. However, since I've noticed you're working very closely with the agents files, I'd prefer you handle this to avoid any merge conflicts between us. I'm mostly fixing some issues that have popped up related to Isaac Gym.

— Reply to this email directly, view it on GitHub https://github.com/AI4Finance-Foundation/ElegantRL/issues/118#issuecomment-1058534273, or unsubscribe https://github.com/notifications/unsubscribe-auth/AST3MB2G4KNW2R2RL7N6KBDU6EY47ANCNFSM5P3XDX3Q . You are receiving this because you were mentioned.Message ID: @.***>

[richardhuo]

Same issue happened to me since yesterday tried to run FinRL via colab: Compare ElegantRL, RLlib, and Stablebaselines3. the elegantrl package was no longer available from that time. I managed to install the package from local but in the end received the same issue of : type object 'AgentXXX' has no attribute 'if_off_policy'

[yizhangliu]

ImportError: cannot import name 'SharePPO' from 'elegantrl.agents.net'

[hmomin]

@richardhuo - that issue should be fixed by now, but you may be running into a different issue where agents cannot be imported from elegantrl.agent. I'll address it tomorrow...

@yizhangliu - that bug should be fixed now.

The original bug should also be fixed now in commit 5d95ce9.

[farzinaghdam]

I am still experiencing this problem. It works fine on Colab. However on my local machine I get File FinRL-Meta/agents/elegantrl_models.py", line 59, in get_model model = Arguments(agent=agent, env=env) File "python3.8/site-packages/elegantrl/train/config.py", line 127, in init self.if_off_policy = agent.if_off_policy # agent is on-policy or off-policy AttributeError: module 'elegantrl.agents.AgentPPO' has no attribute 'if_off_policy'

Thanks

[hmomin]

@farzinaghdam I suggest you try reinstalling ElegantRL on your local machine. I've just pulled the latest commit and all the relevant PPO tests are passing on my machine.

[farzinaghdam]

Thank you for quick reply. I downgraded gym to 0.17.3 (which is the one used on my Colab at the moment) and reinstalled various packages and it works fine now. Thank you.