search

Farama-Foundation / PettingZoo

An API standard for multi-agent reinforcement learning environments, with popular reference environments and related utilities
https://pettingzoo.farama.org
Other
2.45k stars 400 forks source link

Bug: The custom model is not used at the time of training #1200

Open ID-Akash opened 2 months ago

ID-Akash commented 2 months ago

https://github.com/Farama-Foundation/PettingZoo/blob/849414dfcba7f423f4db02bfecc7ef061cccc567/tutorials/Ray/rllib_pistonball.py#L21C7-L21C17

The custom model is not used at the time of training the RL Agent. Instead, the default built-in model is used. This can be verified by simply changing the output dimension of Sequential layer from 512 to any random number and still the training works.

Please suggest the changes required.

ID-Akash commented 2 months ago

Just making the following changes would work:

config = ( PPOConfig() .environment(env=env_name, clip_actions=True) .rollouts(num_rollout_workers=4, rollout_fragment_length=128) .training( train_batch_size=512, lr=2e-5, gamma=0.99, lambda_=0.9, use_gae=True, clip_param=0.4, grad_clip=None, entropy_coeff=0.1, vf_loss_coeff=0.25, sgd_minibatch_size=64, num_sgd_iter=10, model={"custom_model": "CNNModelV2"}, ) .debugging(log_level="ERROR") .framework(framework="torch") .resources(num_gpus=int(os.environ.get("RLLIB_NUM_GPUS", "0"))) )

jjshoots commented 1 week ago

Would you like to make a PR for this?